Networking
Part 2 of 6 · Load balancingL4 vs L7 Proxies — Connection Routing, TLS Termination & Protocol Awareness
TCP/UDP passthrough vs HTTP/gRPC routing; TLS terminate vs passthrough; SNI; HTTP/2 multiplexing implications; when L4 is enough; gRPC/WebSocket affinity needs.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
Where TLS ends
Prefer
L7 terminate at the LB, re-encrypt to backends
Default for HTTP APIs. You get Host/path routing, WAF hooks, and centralized certs. East-west stays encrypted with backend TLS or mTLS.
- ALB / Envoy http{} / NGINX http{} / HAProxy HTTP mode.
- Retries and timeouts are per-request, not per-TCP-conn.
- Trust the LB. That is the product.
Alternative
L4 passthrough (SNI optional)
Required when the threat model forbids the LB seeing plaintext, or when PPS is the SLO. You give up URL routing.
- NLB / NGINX stream{} / HAProxy TCP mode.
- Backends own certificates. Observability is tuples, not paths.
- SNI can pick a backend without decrypting — still not path routing.
Overview
Choose the proxy layer: when TCP/UDP passthrough is enough, when HTTP/gRPC awareness pays for itself, and how TLS terminate vs passthrough changes security and observability.
Cloud map: NLB ≈ L4, ALB ≈ L7. Envoy, NGINX, and HAProxy can do both.
You should be able to:
- Draw passthrough vs terminate on the same whiteboard.
- Refuse "NLB with path-based routing."
- Explain why least-conn on fat HTTP/2 connections lies.
L4 vs L7 practically
- L4 (transport): decisions on IP tuple + port; optionally TCP/UDP awareness (PROXY protocol, idle timeouts).
- L7 (application): parse HTTP/1.1, HTTP/2, HTTP/3, gRPC, WebSocket upgrade.
| Need | L4 | L7 |
|---|---|---|
| Path / Host / header routing | No | Yes |
| End-to-end TLS mandate | Passthrough | Must terminate somewhere |
| WAF / body inspect | No | Yes (after decrypt) |
| Extreme PPS / raw TCP | Yes | Parse cost hurts |
| Per-RPC gRPC LB | No (one TCP conn) | HTTP/2-aware proxy or client-side xDS |
| WebSocket | Forwards bytes | Upgrade + idle timeout + drain |
When L4 is enough: databases, SMTP, binary protocols, or a mesh sidecar that already terminates.
When L7 is required: host/path multi-tenant routing, header canaries, request retries/timeouts, WAF.
Request path
L4 forwards bytes (optional SNI peek). L7 decrypts, reads Host/path, then opens or reuses a backend connection. Do not model decrypt as a labeled self-loop on the L7 actor — it collides with the backend arrow. A note is the honest "this hop thinks."
Sequence
- 1
Client
Step 1 L4 passthrough TLS
- 2
Client → L4 Proxy
1 TCP SYN to VIP port 443
- 3
L4 Proxy → Backend
2 Forward bytes
- 4
L4 Proxy
SNI optional peek
- 5
Backend → Client
3 TLS plus HTTP end to end
- 6
Client
Step 2 L7 terminate TLS
- 7
Client → L7 Proxy
4 TLS handshake to VIP
- 8
L7 Proxy
Decrypt and read Host path
- 9
L7 Proxy → Backend
5 Pooled conn to backend
- 10
Backend → L7 Proxy
6 HTTP response
- 11
L7 Proxy → Client
7 Encrypted to client
Diagrams - step by step
Three small diagrams. Step numbers in the labels give the animation order. The lesson map under Diagram 1 plays those steps.
Diagram 1 - Happy path: passthrough vs terminate
Decisions
- 1
Step 1 Client opens TLS to VIP port 443
- nextStep 2 Proxy layer?
- ?
Step 2 Proxy layer?
- L4Step 3a Forward TCP bytes, optional SNI peek
- L7Step 3b Terminate TLS, read Host and path
- 3
Step 3a Forward TCP bytes, optional SNI peek
- nextStep 4a Backend terminates TLS end to end
- need path routingFailure path - an L4 proxy cannot see the URL
- 4
Step 4a Backend terminates TLS end to end
- nextStep 5 Response returns to the client
- 5
Step 3b Terminate TLS, read Host and path
- nextStep 4b Route per request, re-encrypt or pool to the backend
- 6
Step 4b Route per request, re-encrypt or pool to the backend
- nextStep 5 Response returns to the client
- 7
Step 5 Response returns to the client
- 8
Failure path - an L4 proxy cannot see the URL
The client opens TLS to the VIP. An L4 proxy forwards the TCP bytes, optionally peeking at SNI, and the backend terminates TLS end to end. An L7 proxy terminates TLS, reads Host and path, then routes the request and re-encrypts or pools to the backend. Both paths return the response to the client. An L4 proxy cannot route on the URL.
Lesson map
L4 vs L7 Proxies — Connection Routing, TLS Termination & Protocol Awareness
Diagram 1 walks 7 steps from Step 1 Client opens TLS to VIP port 443 through Step 5 Response returns to the client.
Architecture. Step 1 Client opens TLS to VIP port 443 Ready. Step 2 Proxy layer? Ready. Step 3a Forward TCP bytes, optional SNI peek Ready. Step 3b Terminate TLS, read Host and path Ready. Step 4a Backend terminates TLS end to end Ready. Step 4b Route per request, re-encrypt or pool to the backend Ready. Step 5 Response returns to the client Ready. Failure path - an L4 proxy cannot see the URL Ready
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB A["Step 1 Client opens TLS to VIP port 443 Ready"] B["Step 2 Proxy layer? Ready"] C["Step 3a Forward TCP bytes, optional SNI peek Ready"] E["Step 3b Terminate TLS, read Host and path Ready"] D["Step 4a Backend terminates TLS end to end Ready"] F["Step 4b Route per request, re-encrypt or pool to the backend Ready"] G["Step 5 Response returns to the client Ready"] X["Failure path - an L4 proxy cannot see the URL Ready"] A -->|continues| B B -->|L4| C C -->|continues| D B -->|L7| E E -->|continues| F D -->|continues| G F -->|continues| G C -->|need path routing| X
Diagram 2 - Failure path: gRPC over one connection behind L4
Sequence
- 1
gRPC client → L4 proxy
Step 1 open one HTTP/2 connection
- 2
L4 proxy → Backend 1
Step 2 L4 pins the whole connection to Backend 1
- 3
gRPC client → L4 proxy
Step 3 send 10k RPCs as streams on that connection
- 4
L4 proxy → Backend 1
Step 4 every RPC lands on Backend 1
- 5
Backend 1
Step 5 Backend 1 saturates while Backend 2 idles
- 6
gRPC client
Fix - HTTP/2-aware L7 proxy or client-side per-request LB
The client opens one HTTP/2 connection. L4 pins that whole connection to Backend 1, so all 10k RPCs land there while Backend 2 idles. An HTTP/2-aware L7 proxy, or client-side per-request balancing, splits the RPCs.
Diagram 3 - Decision: L4 or L7, terminate or pass through
Decisions
- ?
Step 1 Need host, path or header routing, retries or canaries?
- yesL7 - terminate TLS at the proxy
- noStep 2 Must TLS stay end to end?
- 2
L7 - terminate TLS at the proxy
- nextRe-encrypt to backends for east-west protection
- Wrong pick under an end-to-end TLS mandateProxy sees plaintext - compliance fails
- ?
Step 2 Must TLS stay end to end?
- yesL4 passthrough, SNI pick at most
- noStep 3 Extreme PPS or non-HTTP protocol?
- 4
L4 passthrough, SNI pick at most
- Wrong pick for gRPCOne long connection pins all RPCs to one backend
- ?
Step 3 Extreme PPS or non-HTTP protocol?
- yesL4 passthrough, SNI pick at most
- noL7 - terminate TLS at the proxy
- 6
Re-encrypt to backends for east-west protection
- 7
One long connection pins all RPCs to one backend
- 8
Proxy sees plaintext - compliance fails
Host, path, or header routing, retries, or canaries mean terminate TLS at an L7 proxy and re-encrypt to the backends. End-to-end TLS, or extreme packet rates and non-HTTP protocols, stay on L4 passthrough with at most an SNI pick. L4 is the wrong pick for gRPC. L7 is the wrong pick when TLS must stay end to end.
TLS terminate vs passthrough
Terminate at the LB: centralized certs; HTTP inspection; you must trust the LB and usually re-encrypt to backends (mTLS or backend TLS). Edge-centric version: TLS termination and Anycast.
Passthrough: true end-to-end crypto; backend owns certs; no URL routing. SNI (Server Name Indication) is in the ClientHello before the handshake finishes — an L4 proxy can pick a cert/backend from the name without full decrypt. That is L4 + metadata, not full L7.
HTTP/2 multiplexing implications
- Connection ≠ request concurrency. Least-conn on TCP mis-estimates fat h2 connections. Prefer outstanding streams (Envoy least-request / P2C). Depth: algorithms.
- Retries: careful with non-idempotent POSTs. Idempotency belongs on the API, not the proxy.
- WebSocket / gRPC streams: long-lived. Draining and idle timeouts matter. Depth: health / drain.
- ALPN: negotiate
h2vshttp/1.1(andh3at an HTTP/3 edge).
Product examples
| Product | Layer | Interview note |
|---|---|---|
| AWS NLB | L4 | Ultra-low latency; PROXY protocol optional; no path routing |
| AWS ALB | L7 | Host/path; HTTP/2; WebSockets; idle timeout is a page |
| Envoy | Both | Maglev, ring hash, outlier detection |
| NGINX | Both | stream{} L4; http{} L7 |
| HAProxy | Both | Excellent TCP mode |
gRPC and WebSocket affinity
gRPC: prefer L7 HTTP/2 per-request LB or client-side xDS. The classic mistake is one forever TCP connection — every RPC lands on one backend.
WebSocket: the connection is sticky for its lifetime. Drain carefully; externalize room/presence state. Same family as SSE / GraphQL subscriptions. Depth: sticky vs stateless.
Operational checklist
- Inventory protocols (HTTP, gRPC, WS, raw TCP, UDP/QUIC).
- Decide the TLS trust boundary (who sees plaintext).
- Measure L7 parse p99 vs capacity before defaulting to "always L7."
- Align idle timeouts with stream lifetime.
- Preserve source IP if required (PROXY protocol /
X-Forwarded-For). - Avoid stacking ALB → NLB → Envoy → sidecar without a reason.
Sandbox: layer chooser (Python)
Press Run. Snippets must be self-contained — no network, files, or native modules.
Same chooser (TypeScript)
Press Run. Snippets must be self-contained — no network, files, or native modules.
Same VIP:443. Path A: client TLS to LB, LB HTTP to backend with mTLS. Path B: client TLS through the LB to the backend. Where do certificates live? Where can you attach a WAF? What does SNI buy you on path B?
Interview Q&A
Can NLB do path routing?
Answer
No. Use ALB or another L7 proxy. NLB is L4 (tuple/port).
Why re-encrypt after ALB terminates TLS?
Answer
East-west protection and compliance. The VIP hop is not the only hop that can be tapped. Backend TLS or mTLS is the usual answer.
What is SNI routing?
Answer
Pick a certificate or backend from the ClientHello server name without a full decrypt. Useful on L4. It is not Host/path routing.
HTTP/2 plus least-conn pitfall?
Answer
One TCP connection can carry many streams. Conn-count least-conn underestimates fat clients. Balance on outstanding requests / P2C.
L4 plus Envoy sidecar vs ALB alone?
Answer
The mesh gives identity (SPIFFE/mTLS), locality, and per-RPC retries. ALB alone is a regional L7 VIP. They solve different hops — do not stack them by habit.
WebSocket through ALB?
Answer
Supported. Tune idle timeouts to the longest quiet period. Plan connection draining on deploy.
Classic gRPC load-balancing mistake?
Answer
The client opens one TCP connection for the process lifetime, so every RPC hits one backend. Fix with an HTTP/2-aware proxy (per-request) or client-side xDS.