Operating systems
Part 5 of 6 · Linux I/O Models & Event LoopsReactor vs Proactor - How libuv, Nginx, Netty, Tokio & the Go Netpoller Work
Two design patterns turn OS I/O primitives into a programming model. A **reactor** waits for *readiness* (epoll, kqueue, select) and dispatches to a handler that then performs the non-blocking read or write itself: "socket 7 is readable, go read it". A **proactor** starts an *operation* and dispatches the *completion*: "your read on socket 7 finished, here are 4,096 bytes". Linux servers are mostly reactors because epoll is readiness-based; Windows IOCP and Linux io_uring are completion-based, so runtimes built on them are proactors. Every major runtime is some arrangement of these: **libuv** (Node) is a single-threaded reactor plus a threadpool that fakes completions for files; **Nginx** runs one reactor per worker process; **Netty** runs one reactor per event-loop thread; **Tokio** runs reactor-driven futures on a work-stealing pool; **Go's netpoller** hides a reactor under goroutines so your code looks blocking.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
Question ladder
L1
Define reactor and proactor in one sentence each.
Answer
A reactor says the fd is ready and you do the I/O. A proactor says the operation finished and hands you the bytes.
L2
Which OS APIs are which?
Answer
select, poll, epoll, and kqueue are readiness, so reactors. IOCP and io_uring are completion, so proactors.
L3
Is Node single-threaded?
Answer
JavaScript and the loop are one thread. libuv also has a threadpool for files, getaddrinfo, and some crypto, plus V8 helper threads.
L4
What does a goroutine do on EAGAIN?
Answer
The runtime parks it, registers the fd with the netpoller, and runs something else. When epoll reports the fd, the goroutine retries the read.
L5
Why does Netty pin a channel to one event loop?
Answer
Callbacks for that connection stay ordered and lock-free. A hot connection cannot use a second core.
L6
What is the memory problem with a naive proactor?
Answer
Each pending read holds a buffer. Idle connections pin that memory until you use provided buffers or a zero-byte readiness read.
L7
How do work-stealing and pinning differ?
Answer
Tokio can move tasks across workers and needs Send. Nginx and Netty keep a connection on one thread and then need a way to balance accepts.
Failure modes
Blocking callback stalls the reactor
The loop is inside a sync file read, a DNS lookup, or a long JSON parse. Every connection on that loop misses timers and reads.
Proactor buffer pin on idle sockets
Posted reads hold memory for connections that may never send.
Shared mutable state across loops
Loop-per-core only stays lock-free if a connection does not bounce between workers.
Misconceptions
Node has no threads.
The JavaScript heap is one loop. The libuv pool and V8 background threads are real threads.
Go does blocking I/O with one OS thread per goroutine.
The netpoller parks the goroutine. A truly blocking syscall hands off the OS thread so other goroutines continue.
A proactor is just a reactor with different names.
The handler in a proactor does not call read. The kernel already did.
Interviewer traps
Drawing one box labeled 'event loop' for every runtime.
Say how many loops, whether a connection is pinned, and where blocking work goes.
Claiming cancellation is free on io_uring or IOCP.
The buffer and the op are in flight. You cancel and then wait for the completion that says the cancel finished.
Design scenario
Same prompt for every reader.
Requirements
Say who is a reactor, who can look like a proactor, where the blocking library must go, and why the 16 KiB posts are expensive.
Traffic / scale
Mixed RPC plus a large idle connection set.
Latency
A blocking call on a single loop adds its duration to every other client on that loop.
Consistency
A channel pinned to one loop sees events in order. Work-stealing must not break that assumption.
Availability
One stuck loop fails the clients hashed onto it. A buffer pin can OOM before traffic arrives.
Failure assumptions
- A dependency exposes only a blocking API.
- Most sockets are idle.
- More than one core is available.
Constraints
- Do not run the blocking library on the socket loop.
- Do not post a large buffer per idle connection without a pool.
Prompt
You are comparing a Node process, an Nginx worker, a Netty server with a boss group and a worker group, a Tokio multi-thread runtime, and a Go service. One of them must call a blocking stats library. Another holds 100,000 idle sockets and is considering posting a 16 KiB read on each.
API
What does the handler receive in each pattern?
Data
How much memory is 100,000 posted 16 KiB buffers?
Architecture
Which runtime parks a goroutine, and which pins a channel?
Who performs the read
Prefer
A reactor on epoll for sockets, with blocking work off the loop
The handler reads only after readiness. Files, DNS, and CPU go to a pool or another loop. One channel stays on one thread so per-connection state needs no lock.
- Idle sockets do not hold posted buffers.
- Nginx and Netty scale by adding loops, not by sharing one fd across threads.
- Go hides the park so the code can look blocking.
Alternative
A proactor that posts a full buffer for every idle socket
The kernel fills the buffer, which is what IOCP and io_uring do well. Posting 16 KiB per idle connection pins memory before any byte arrives.
- Cancellation has to wait for the in-flight op.
- Provided buffer rings and zero-byte reads exist to avoid that pin.
- A blocking callback on a reactor still stalls every fd on that loop.
Ready, then read - or submit, then reap
The reactor column performs the read. The proactor column receives a filled buffer.
- 1
Reactor: wait, then read
Register the fd, block in epoll_wait, and let the handler call non-blocking read. - 2
Reactor: the handler owns the bytes
Processing and the next write decision happen after that read. A blocking call here stalls the loop. - 3
Proactor: submit the read with a buffer
The kernel performs the read. A completion lands on a queue. - 4
Proactor: the handler receives the buffer
The loop dequeues the completion and starts the next op. The buffer was committed at submit.
Reactor vs proactor
| Reactor | Proactor | |
|---|---|---|
| Kernel tells you | fd is ready | operation completed |
| Who performs the I/O | Your handler (non-blocking read/write) | The kernel / OS, into the buffer you supplied |
| Buffer ownership | You allocate at read time; only when data is ready | Committed at submit; held until completion (memory per pending op) |
| OS fit | epoll, kqueue, select, poll | IOCP (Windows), io_uring (Linux), POSIX AIO |
| Typical runtimes | libuv sockets, Nginx, Netty NIO, mio/Tokio, Redis ae, Python asyncio selector loop | Boost.Asio on Windows, .NET sockets on Windows, asyncio ProactorEventLoop, tokio-uring, Netty io_uring transport |
| Cancellation | Easy: just stop reading | Harder: op in flight owns the buffer, must cancel and wait |
Architecture
Reactor
- 1
1. Register fd for readable
- next2. Loop waits in epoll_wait
- 2
2. Loop waits in epoll_wait
- next3. fd ready, dispatch handler
- 3
3. fd ready, dispatch handler
- next4. Handler calls non-blocking read
- 4
4. Handler calls non-blocking read
- next5. Handler processes bytes, maybe queue write
- 5
5. Handler processes bytes, maybe queue write
Proactor
- 6
1. Start async read with buffer
- next2. Kernel performs read into buffer
- 7
2. Kernel performs read into buffer
- next3. Completion posted to queue
- 8
3. Completion posted to queue
- next4. Loop dequeues completion
- 9
4. Loop dequeues completion
- next5. Handler gets filled buffer, starts next op
- 10
5. Handler gets filled buffer, starts next op
Lesson map
Reactor vs Proactor - How libuv, Nginx, Netty, Tokio & the Go Netpoller Work
>-
Architecture. Architecture
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB r1["1. Register fd for readable"] p1["1. Start async read with buffer"] r2["2. Loop waits in epoll_wait"] p2["2. Kernel performs read into buffer"] r3["3. fd ready, dispatch handler"] p3["3. Completion posted to queue"] r4["4. Handler calls non-blocking read"] p4["4. Loop dequeues completion"] r5["5. Handler processes bytes, maybe queue write"] p5["5. Handler gets filled buffer, starts next op"] r1 -->|continues| r2 r2 -->|continues| r3 r3 -->|continues| r4 r4 -->|continues| r5 p1 -->|continues| p2 p2 -->|continues| p3 p3 -->|continues| p4 p4 -->|continues| p5
A tiny reactor with timers (runnable)
This is the core of libuv, asyncio and Netty in about 30 lines: run due timers, wait for I/O no longer than the next timer deadline, dispatch ready handlers.
# A tiny reactor (the libuv/Nginx/Netty-NIO shape): one loop, readiness events,
# timers, and callbacks. The handler does the read itself once told "ready".
import heapq, itertools, selectors, socket, time
class Reactor:
def __init__(self):
self.sel = selectors.DefaultSelector()
self.timers, self.seq = [], itertools.count()
self.log = []
def on_readable(self, sock, cb):
sock.setblocking(False)
self.sel.register(sock, selectors.EVENT_READ, cb)
def call_later(self, delay, cb):
heapq.heappush(self.timers, (time.monotonic() + delay, next(self.seq), cb))
def run(self, until):
while not until():
# Phase 1: run due timers. Phase 2: wait for I/O no longer than next timer.
now = time.monotonic()
while self.timers and self.timers[0][0] <= now:
heapq.heappop(self.timers)[2]()
timeout = max(0, self.timers[0][0] - now) if self.timers else 0.1
for key, _ in self.sel.select(timeout):
key.data(key.fileobj) # dispatch: handler performs the read
r = Reactor()
pairs = [socket.socketpair() for _ in range(3)]
replies = []
def handler(sock):
data = sock.recv(64) # ready => non-blocking read succeeds
r.log.append(f"read {data.decode()}")
sock.send(data[::-1])
r.sel.unregister(sock)
for i, (_, srv) in enumerate(pairs):
r.on_readable(srv, handler)
r.call_later(0.02, lambda: r.log.append("timer fired (20ms)"))
for i, (cli, _) in enumerate(pairs):
cli.send(f"req{i}".encode())
r.run(until=lambda: len(r.log) == 4)
print("\n".join(r.log))
print("replies:", [cli.recv(64).decode() for cli, _ in pairs])Output:
read req0
read req1
read req2
timer fired (20ms)
replies: ['0qer', '1qer', '2qer']The proactor shape (simulated, runnable)
// Proactor shape (IOCP, io_uring, Boost.Asio on Windows): you START an operation
// with a buffer, and the completion handler receives the finished result.
// Simulated device; compare with the reactor where the handler does the read.
type Completion = { op: string; bytes: number; data?: string };
class Proactor {
private queue: Array<() => Completion> = [];
private handlers: Array<(c: Completion) => void> = [];
// Initiate: the "kernel" owns the buffer until completion.
asyncRead(fd: number, size: number, onDone: (c: Completion) => void) {
this.queue.push(() => ({ op: `read fd=${fd}`, bytes: size, data: `payload-${fd}` }));
this.handlers.push(onDone);
}
asyncWrite(fd: number, data: string, onDone: (c: Completion) => void) {
this.queue.push(() => ({ op: `write fd=${fd}`, bytes: data.length }));
this.handlers.push(onDone);
}
// Event loop: dequeue completions (like GetQueuedCompletionStatus / reaping a CQ).
run() {
while (this.queue.length) {
const work = this.queue.shift()!; const h = this.handlers.shift()!;
h(work());
}
}
}
const p = new Proactor();
for (const fd of [7, 8]) {
p.asyncRead(fd, 4096, (c) => {
console.log(`completed ${c.op}: ${c.bytes}B data=${c.data}`); // data already in buffer
p.asyncWrite(fd, c.data!.toUpperCase(), (w) => console.log(`completed ${w.op}: ${w.bytes}B`));
});
}
p.run();
console.log("reactor: 'fd is readable, you read it'; proactor: 'your read finished, here are the bytes'");Output:
completed read fd=7: 4096B data=payload-7
completed read fd=8: 4096B data=payload-8
completed write fd=7: 9B
completed write fd=8: 9B
reactor: 'fd is readable, you read it'; proactor: 'your read finished, here are the bytes'Expectedcompleted read fd=7: 4096B data=payload-7 completed read fd=8: 4096B data=payload-8 completed write fd=7: 9B completed write fd=8: 9B reactor: 'fd is readable, you read it'; proactor: 'your read finished, here are the bytes'
Press Run. Snippets must be self-contained — no network, files, or native modules.
How real runtimes are put together
libuv / Node.js. One loop thread with phases: timers, pending callbacks, idle/prepare, poll (epoll/kqueue wait, with timeout set by the next timer), check (setImmediate), close callbacks. JavaScript promises (microtasks) drain between callbacks. File system operations, dns.lookup (getaddrinfo), and CPU-heavy crypto/zlib run on the libuv threadpool (default 4 threads, UV_THREADPOOL_SIZE, max 1024). So Node is a reactor for sockets and a proactor emulation for files.
Nginx. A master process plus N worker processes (usually one per core). Each worker is a single-threaded reactor over epoll, edge-triggered, handling thousands of connections. Optional thread pools offload blocking file reads. No shared state between workers; reuseport or accept_mutex controls how connections spread.
Netty (JVM). A "boss" EventLoopGroup accepts, a "worker" group holds N event loops (often 2 x cores), each a thread with its own selector (NIO) or native epoll. A channel is pinned to one loop for life, so handlers never need locks for per-connection state. Blocking work must go to a separate executor.
Tokio (Rust). Futures are polled by a scheduler; when a socket isn't ready, the future registers a waker with the reactor (mio over epoll/kqueue/IOCP) and returns Pending. The default multi-thread runtime work-steals tasks across cores. spawn_blocking moves blocking code to a separate pool.
Go netpoller. Goroutine calls conn.Read; the runtime tries a non-blocking read, gets EAGAIN, parks the goroutine and registers the fd with the netpoller (epoll/kqueue/IOCP). When the fd is ready the goroutine becomes runnable on some P. Blocking syscalls (e.g., file I/O, cgo) hand the OS thread off so other goroutines keep running. You write blocking code; the runtime is a reactor underneath.
Python asyncio. SelectorEventLoop is a reactor (epoll/kqueue). On Windows the default is ProactorEventLoop over IOCP. uvloop swaps in libuv for speed.
What happens if you pick the other pattern
- Reactor where you wanted proactor: on Windows, readiness APIs (select) don't scale, so frameworks moved to IOCP. On Linux with heavy file I/O, a reactor needs a threadpool for disk, which becomes a hidden bottleneck.
- Proactor where you wanted reactor: each pending read needs a committed buffer. 100k idle connections x 16 KB = 1.6 GB of buffers waiting for data. io_uring mitigates with provided buffer rings (the kernel picks a buffer only when data arrives); IOCP apps use zero-byte reads as a readiness trick.
- Single loop vs many loops: one loop (Node, Redis) means no locking and simple ordering but one core of I/O; scale by processes. Loop-per-core (Nginx, Netty, Seastar) scales across cores but needs connection distribution and no shared mutable state.
- Work-stealing vs pinned: Tokio's work stealing balances uneven load but moves tasks across threads (Send bounds, cache misses). Netty/Seastar pinning keeps cache locality and lock-free per-connection state but can leave one core hot.
Pros and cons
| Design | Pros | Cons |
|---|---|---|
| Single-threaded reactor (Node, Redis) | No locks, deterministic ordering, tiny overhead | One core; any slow callback stalls all |
| Reactor per core (Nginx, Netty) | Scales with cores, per-connection state stays lock-free | Accept balancing, cross-loop communication needs queues |
| Work-stealing async (Tokio) | Good load balance, high throughput | Tasks must be thread-safe; harder latency reasoning |
| Proactor (IOCP, io_uring) | Fewer syscalls, real async files | Buffer commitment, complex cancellation, platform-specific |
| Green threads over reactor (Go) | Blocking-style code, scales | Runtime magic hides costs; still need limits and timeouts |
Interview Q&A
Explain reactor vs proactor in one sentence each.
Answer
A reactor notifies you when an fd is ready and your handler does the I/O; a proactor performs the I/O for you and notifies you when it completed, handing you the filled buffer.
Is Node.js single-threaded?
Answer
The JavaScript and the event loop are single-threaded; libuv also runs a threadpool for file I/O, DNS lookups and some crypto/zlib, plus V8 helper threads for GC and compilation. Socket I/O is handled by epoll/kqueue/IOCP on the loop thread.
Walk through what happens when a goroutine reads from a TCP connection with no data.
Answer
The runtime does a non-blocking read, gets EAGAIN, parks the goroutine on the fd's poll descriptor and registers it with epoll. The scheduler runs other goroutines. When epoll reports the fd readable (checked by findrunnable or sysmon), the goroutine is made runnable, retries the read and gets the data.
Why does Netty pin a channel to one event loop?
Answer
All callbacks for that connection run on one thread, so handlers can keep per-connection state without locks and events are processed in order. The cost is that a hot connection can't be spread across cores.
What's the main memory problem with proactor designs for many idle connections, and how is it solved?
Answer
You must post a buffer per pending read, so idle connections pin memory. Solutions: kernel-selected buffers from a shared pool (io_uring provided buffers), zero-byte reads as a readiness signal (IOCP), or small initial buffers.
What are the libuv poll phases a Node candidate should be able to name?
Answer
Timers, pending callbacks, idle and prepare, the poll wait (timeout set by the next timer), check callbacks for setImmediate, and close callbacks. Promise microtasks drain between callbacks. File and DNS work is the threadpool, not the poll phase.
Why does asyncio use a different loop on Windows?
Answer
SelectorEventLoop is a reactor over epoll or kqueue. Windows servers scale with IOCP, so the default there is ProactorEventLoop. uvloop swaps the Unix reactor for libuv.
What is a zero-byte read on IOCP?
Answer
A posted read of length zero completes when the socket becomes readable, without pinning a full buffer. It is a readiness trick on a completion API. io_uring's provided buffer ring is the other fix: the kernel picks a buffer only when data arrives.
Check yourself
Pick one runtime you use. Write who waits, who reads, how many loops you run, and what happens if a handler calls a blocking SDK.
Elsewhere in the library
These pages stay as they are. This lesson only points at them: the JavaScript event loop, Python asyncio, Rust and Tokio, Go goroutines.