The idea: two ways to wait for I/O

A server spends its life waiting: for bytes on a socket, for a block from the disk. Linux has two main interfaces for waiting on many things at once. epoll tells you when an fd is ready, and then you do the read yourself. io_uring takes the read itself, does it in the background, and tells you when it is done. The difference shows up as the number of system calls, each one a trip from user space into the kernel and back.

Left, epoll: the app calls epoll_wait, sleeps, gets back the list of ready sockets s1 to s4, then calls read on each one, five system calls in all. Right, io_uring: the app writes four SQEs into the submission ring, calls io_uring_enter once, the kernel copies the data into the app's buffers and writes four CQEs into the completion ring, which the app reads without a system call
epoll answers "which fds can I read?" and leaves one read() per fd to you; io_uring takes the reads and hands back completions, one syscall for the whole batch.

The canvas runs the same work twice: epoll on the left, io_uring on the right. Each side is split into user space, the boundary between the app and the kernel, the kernel, and the device (a network card or an SSD). Under each side are its counters: system calls, user↔kernel crossings (two per system call), context switches (two each time the thread sleeps) and the time spent waiting. Setup calls such as epoll_ctl(ADD) and io_uring_setup happen once and are not counted.

epoll: readiness, then read()

The app adds its sockets to an epoll instance once (epoll_ctl), then loops on epoll_wait. The thread sleeps until a socket gets data; the kernel puts that socket on the ready list and epoll_wait returns the ready fds. The data is still in the kernel's receive queues. The app now calls read() on each ready fd, and each call copies the bytes into its buffer. Four ready sockets cost 1 + 4 = 5 system calls (Demo 1), and three rounds cost 15 (Demo 3). See epoll for the interest list, ready list and edge vs level triggering.

io_uring: two rings in shared memory

io_uring (Linux 5.1, 2019) sets up two ring buffers that the app maps into its own memory with mmap, so the app and the kernel both read and write them directly:

ringwritten byread byone entry says
SQ (submission queue)the app (moves the tail)the kernel (moves the head)SQE: opcode (IORING_OP_RECV, IORING_OP_READ, …), fd, buffer, length, user_data
CQ (completion queue)the kernel (moves the tail)the app (moves the head)CQE: user_data (which request), res (bytes or −errno)

Writing an SQE and reading a CQE are plain memory writes and reads, not system calls. The head and tail are counters that only grow; the slot is the counter modulo the ring size, so in Demo 3 the second round lands in slots 4–7 and the third wraps around to slots 0–3. The kernel does the read itself and copies the data straight into the buffer named in the SQE: the app never calls read().

The SQ and CQ rings of 8 slots each. The app writes SQEs at the SQ tail, the kernel takes them from the SQ head. The kernel writes CQEs at the CQ tail, the app reads them from the CQ head. Each CQE carries the user_data of its SQE and the result in bytes
Each side only moves its own index: the app owns the SQ tail and the CQ head, the kernel owns the SQ head and the CQ tail.

Batching and SQPOLL

One io_uring_enter(fd, to_submit, min_complete, IORING_ENTER_GETEVENTS) both submits every new SQE and waits for completions. Four reads cost one system call instead of five (Demo 1). If only one socket gets data, epoll still pays 2 calls and io_uring 1, and the other three receives stay armed in the kernel (Demo 2). Across three rounds the app re-arms the receives and waits in the same call: 3 syscalls against 15.

With IORING_SETUP_SQPOLL a kernel thread keeps polling the SQ ring, so new SQEs are picked up with no system call at all, and the app finds completions by checking the CQ ring (Demo 4: 0 syscalls, 0 context switches). The price is a CPU core that spins; after sq_thread_idle without work the thread sleeps and the app has to wake it with a syscall.

workloadepollio_uringio_uring + SQPOLL
4 sockets with data5 syscalls1 syscall0
1 of 4 sockets with data210
3 rounds × 4 sockets1530
4 file blocks4 blocking reads, 400 µs1, 100 µs0, 100 µs

Regular files: where epoll cannot help

epoll only works for things that can be "not ready": sockets, pipes, terminals, eventfds. A regular file is always ready, so epoll_ctl(ADD) on it fails with EPERM and read() simply blocks while the disk works. Four blocks not in the page cache mean four reads in a row, each one a sleep: 400 µs here (Demo 5). That is why Node.js (libuv) sends file reads to a thread pool. io_uring does real asynchronous file I/O: all four reads go to the SSD together and finish in about the time of one.

Why fewer system calls matter, and the catches

A system call is not free: the CPU switches privilege level, saves registers and, since the Spectre and Meltdown mitigations, often flushes or switches page tables (see what a system call does). For a server doing millions of small reads per second, that overhead is a large part of the work. io_uring cuts it further with registered files and fixed buffers (set up once so the kernel does not look them up on every request) and multishot receive, where one SQE keeps producing a CQE for every packet so nothing has to be re-armed.

The catches: io_uring is a large attack surface, and several kernel bugs have come through it, so some environments disable it (Android apps, ChromeOS, some container runtimes and cloud defaults). It needs a recent kernel, and its API is harder to use well; most programs use it through liburing or a runtime such as Tokio, Netty or libuv. epoll remains the simple, portable choice for socket servers, as in the Node.js event loop.

Node.js still sends file reads to libuv's thread pool, because epoll can't wait for files; see Node.js under the hood.

For where both fit next to blocking threads and EAGAIN polling, see Blocking vs Non-blocking vs Async I/O.

What the page leaves out

Edge vs level triggering, EPOLLONESHOT and EPOLLEXCLUSIVE; linked SQEs, timeouts and cancellation; the io-wq worker threads io_uring uses for work that cannot be done asynchronously; IORING_SETUP_IOPOLL for NVMe polling; what happens when the CQ ring overflows; real timings (the numbers here are only a rough scale); and the hardware side of how the data gets into memory, shown in interrupt-driven I/O.