Up until now, we've mostly dealt with one connection at a time. That's easy. You accept() a client, you talk to them, you close the socket. But real-world servers—like a chat room or a load balancer—can't just freeze the entire process while waiting for one slow client to type a message. I've seen a lot of developers try to solve this by spawning a new thread for every single connection. While that works for five or ten users, it falls apart completely when you hit a thousand. Context switching will eat your CPU alive before your actual logic even runs.

The "Busy-Wait" Loop and why it fails

The most naive way to handle multiple connections without threads is to set all your sockets to non-blocking mode and just loop through them constantly, calling recv() on each one to see if any data has arrived. It looks something like this in your head: "Is client A talking? No. Client B? No. Client C? Yes!"

while (running) {
    for (int i = 0; i < max_clients; i++) {
        if (clients[i] != -1) {
            int n = recv(clients[i], buffer, sizeof(buffer), MSG_DONTWAIT);
            if (n > 0) {
                // Process data
            }
        }
    }
}

The problem here is that you're burning 100% of a CPU core just to ask the kernel "Anything yet?" a million times a second. It's incredibly inefficient. You're essentially polling the hardware manually. We need a way to tell the kernel: "Put this thread to sleep, and wake me up only when at least one of these sockets actually has something for me to read."

Level-Triggering with poll()

This is where poll() comes in. Instead of you looping over the sockets, you give the kernel a list of file descriptors you're interested in. The poll() call blocks your program, meaning your process consumes zero CPU while waiting. When one or more sockets become "ready" (meaning there's data in the receive buffer), poll() returns.

However, there's a catch. poll() is O(N). Every time it returns, it doesn't tell you which socket is ready; it just tells you something happened. You still have to loop through your entire array of pollfd structures and check the revents flag for each one. If you have 10,000 connections and only one person sends a message, you're still iterating 10,000 times. For a small number of connections, this is perfectly fine and very portable, but for high-performance systems, it's a bottleneck.

Scaling to thousands with epoll()

If you're developing on Linux, you should reach for epoll() when you expect a high volume of connections. I think of epoll as an "event interest list" maintained inside the kernel. Instead of passing a full list of sockets to the kernel every single time you call the function (like poll does), you create an epoll instance and add or remove sockets from it as clients connect and disconnect.

The magic happens during the wait. When you call epoll_wait(), the kernel doesn't make you scan a list. It returns an array containing only the file descriptors that actually triggered an event. If you have 10,000 connections and only two are active, epoll_wait() returns a list of size two. It's O(1) in terms of the number of monitored connections, making the performance cost independent of how many idle clients are connected.

The trade-off is complexity. epoll is Linux-specific (BSD/macOS uses kqueue), and the API is more involved—you have to manage the epoll file descriptor itself. But if you're building something that needs to scale, the jump from poll to epoll is the single most important optimization you can make in your networking stack.

📋 Practical Task

Build a Multi-Client Broadcast Server using epoll

Your task is to create a server that can handle multiple simultaneous TCP connections using epoll(). The server should behave as a simple broadcast hub: whenever any client sends a message, the server should broadcast that message (including the sender's identifier) to all other connected clients.

• Create an epoll instance using epoll_create1().
• Set your listening socket to non-blocking and add it to the epoll interest list for EPOLLIN events.
• Use epoll_wait() in a loop to handle events.
• When the listening socket triggers EPOLLIN, accept() the new connection, set it to non-blocking, and add it to the epoll list.
• When a client socket triggers EPOLLIN, read the data and send() it to all other active file descriptors currently managed by your server.
• Ensure that when recv() returns 0, you remove the client from the epoll list using epoll_ctl() with EPOLL_CTL_DEL and close the socket.