How Many Concurrent Online Users Can a Single JuggleIM Node Support?
Lately, customers often ask about our IM performance, usually along these lines:
"How many long connections can a single IM machine of yours support?"
"How many concurrent online users can one machine handle?"
… and so on.
I usually reply, "A single JuggleIM node can easily handle a million connections, no big deal 😁", which often gets me a snort — "You're bullshitting..." 😂
It turns out many people think a single machine can support at most around 60,000 connections. Some even add authoritatively, "That's limited by the machine's port range — it's a physical limit."
Today, let's discuss what really limits the number of long connections a single machine can maintain.
IM long connections mostly use TCP. A single TCP connection can be uniquely identified by a 4-tuple: {source IP, source port, destination IP, destination port}.
For the server, it usually only listens on one port, like 80 or 443. For the client, however, the maximum number of connections a device can initiate is around 60,000+, which is indeed limited by the port range (1024~65535). Note: this is a client-side limit, not a server-side limit.
So, what factors limit the number of concurrent connections on the server?
1. Maximum number of file handles (file descriptors): A long connection occupies one file descriptor (fd). For a Linux system, the number of long connections it can maintain is directly tied to the number of file handles. Linux's maximum handle count is limited in two places:
- User-level limit: For Linux, the default is 1024. You can check it with
ulimit -n. - System-level limit: The default is around 38,000, which can be checked at
/proc/sys/fs/file-max.
Both of these limits can be modified. In the example below, we've raised the upper limit to 1 million:
# Modify the user-level limit. The maximum nofile depends on the fs.nr_open variable, which can be modified in /etc/sysctl.conf.
echo "* soft nofile 1000000" >> /etc/security/limits.conf
# Modify the system-level limit
echo "fs.file-max = 1000000" >> /etc/sysctl.conf
2. Machine memory size: In the example above, we've raised the file-handle limit to 1 million, so in theory 1 million long connections can be maintained simultaneously. But whether that many can actually be established is also limited by the machine's memory. So, how much memory does a long connection consume? Take a TCP long connection as an example — its memory footprint consists of:
- Socket structures: mainly two core structures — 1)
struct socket, which stores the file descriptor, state, and protocol pointer, taking about 100–200 bytes; 2)struct tcp_sock, the TCP control block, containing sequence numbers, window size, retransmission timers, optional parameters, etc., taking about 300–500 bytes. - Buffers: For an idle connection, buffers can be mostly ignored. During data transfer, there are receive and send buffers, with Linux's default typically being 4KB each.
- Other auxiliary structures: such as route caches, firewall rules, etc., taking an extra tens to hundreds of bytes.
So, a single TCP long connection consumes less than 4KB when idle, and between 4KB and a few MB during data transfer, depending on buffer settings. Therefore, if we only look at how many long connections a single machine can keep (idle long connections), an 8GB-memory machine can simultaneously establish (8 x 10^9) / (4 x 10^3) ≈ 2 million long connections.
Conclusion
Bragging about idle connections is meaningless. For an IM system, what matters is the maximum number of connections that can be sustained during real business. Generally, an 8GB-memory server can stably maintain at least 100,000 active connections. With a 64GB+ memory server, that goes to over 1 million active connections.
Of course, real production systems are more complex. Network bandwidth, concurrent request processing capability, and more will all affect the actual concurrent connection count. Therefore, the ultimate way to improve the overall capacity of the system is to design distributed architecture and give the system the ability to scale out smoothly.