Networking
Part 3 of 6 · Sidecar & Service MeshData-Plane Proxies — Envoy, nginx, Linkerd-proxy & What They Own
The data plane is the process that touches every request. Envoy, nginx, and Linkerd-proxy play the same role with different L7 depth, memory, and ops. Interviews want what lives in the proxy vs the app — not an Envoy product tutorial.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
What touches every request
Prefer
Proxy owns cross-cutting policy; app owns business spans
Transport identity, retry budgets, and request metrics live out of process so languages do not skew. The app still traces business logic. Listeners, filters, and clusters are the same abstract model in different YAML dialects.
- mTLS terminate/originate with rotation — often SPIFFE or a platform CA.
- Outlier ejection is passive; active probes live in the LB health-check lesson.
- gRPC needs an HTTP/2-aware L7 proxy; raw L4 will not give route-level retries.
Alternative
Stuff retries into every SDK, or treat nginx vs Envoy as religions
Language skew returns. Edge gateways (Kong, ALB, API Gateway) terminate TLS at the perimeter — they are not every-pod east-west peers.
- Double retries (app plus proxy) amplify outages.
- Filter order matters: auth before route, decode before body rate-limit.
- WebSockets and long streams: retries mid-stream are unsafe.
One request through the data plane
Same abstract hop in Envoy, nginx, or Linkerd-proxy.
- 1
Accept on a listener
Bind port and protocol. Capture already stole the socket. - 2
Run the filter chain
TLS, decode, authz, route match, maybe fault injection. - 3
Pick a cluster
Logical upstream plus LB policy. Algorithm depth is the other cluster. - 4
Skip outliers
Passive ejection from real traffic. Active health is a different signal. - 5
Emit telemetry
Counters, access logs, trace headers. Business spans stay in the app.
Overview
The data plane is the process that touches every request. Envoy is the most cited mesh sidecar, but nginx (and OpenResty patterns), Linkerd-proxy (Rust, ultralight), HAProxy, and others play the same role. Interviews want: what lives in the proxy vs the app, and how proxies differ on L7 features, memory, and ops.
Do not confuse with edge gateways: Kong, AWS ALB, and API Gateway products also terminate TLS and route — but they sit at the edge, not as every-pod east-west peers. Depth: mesh vs gateway.
What a data-plane proxy typically owns
| Concern | In the proxy | Still in the app |
|---|---|---|
| Transport security | mTLS terminate/originate, cert rotation | Business authz, user sessions |
| Resilience | Retries, timeouts, outlier, rate limits | Idempotency of handlers |
| Observability | Metrics, access logs, trace header propagate | Business spans and events |
| L7 routing | Path/header/method, canary weights | Domain-specific dispatch |
| Codecs | HTTP/1.1, HTTP/2, gRPC, TCP/UDP passthrough | Serialization of payloads |
mTLS mechanics and SPIFFE IDs: s2s auth. Do not re-teach OAuth client credentials here.
Comparative snapshot (honest, not a bake-off)
| Proxy | Wins | Loses |
|---|---|---|
| Envoy | Rich L7 filter chain, xDS-native, hot restart story | Higher baseline CPU/mem |
| nginx | Reverse proxy, static, simple L7; Lua/modules | Less mesh-native discovery unless wrapped |
| Linkerd-proxy | Purpose-built ultralight mesh; strong mTLS defaults | Narrower surface than Envoy |
| Ambient / node proxy | Shared datapath, less per-pod tax | Different isolation |
Decisions
- 1
1 Listener bind
- next2 Filter chain
- 2
2 Filter chain
- next3 Cluster pick
- next7 Metrics and traces
- 3
3 Cluster pick
- next4 Endpoint healthy?
- ?
4 Endpoint healthy?
- no5 Eject outlier
- yes6 Upstream call
- 5
5 Eject outlier
- 6
6 Upstream call
- 7
7 Metrics and traces
Lesson map
Data-Plane Proxies — Envoy, nginx, Linkerd-proxy & What They Own
The data plane is the process that touches every request. Envoy, nginx, and Linkerd-proxy play the same role with different L7 depth, memory, and ops. Interviews want what lives in the proxy vs the app — not an Envoy product tutorial.
Architecture. Architecture
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB l["1 Listener bind"] f["2 Filter chain"] c["3 Cluster pick"] o["4 Endpoint healthy?"] l -->|1 Listener bind to 2 Filter chain| f f -->|2 Filter chain to 3 Cluster pick| c c -->|3 Cluster pick to 4 Endpoint healthy?| o
Listeners + filters + clusters (or nginx upstream + location; Linkerd outbound policy) are the same abstract model — different YAML dialects. How that YAML arrives: control plane.
Protocol ownership
Name the protocol before naming the proxy.
| Protocol | Proxy can do | Do not expect |
|---|---|---|
| TCP passthrough | mTLS wrapper, connection metrics | Route-level HTTP retries |
| HTTP / gRPC | Route, timeout, retry, fault injection | Safe retries on non-idempotent POST |
| WebSockets / streams | Drain and idle timeouts | Mid-stream retry |
L4 vs L7 at the edge is L4 vs L7 proxies. This page does not recap the matrix, Maglev, or P2C.
Retries, timeouts, outlier vs health
- Prefer proxy timeouts as the outer bound. App timeouts should be ≤ proxy or you double-retry.
- Retry only idempotent methods / explicit retry budgets — the proxy can amplify outages.
- Outlier = passive from real traffic. Active probes = health checks in the LB health-check lesson. Different signals; they complement, they do not replace each other.
Sandbox: outlier ejection (Python)
Proxy-agnostic sketch. Five consecutive fails eject a; b stays in the pool.
Press Run. Snippets must be self-contained — no network, files, or native modules.
Same idea: listeners and clusters (TypeScript)
Press Run. Snippets must be self-contained — no network, files, or native modules.
Pitfalls
Inbound 15001. Draw TLS, HTTP decode, route to cluster payments, outlier max 5. Now the protocol is raw TCP to a database. What filters disappear? Where does mTLS still belong?
Interview Q&A
What must never live only in the app if you chose a mesh?
Answer
Cross-cutting mTLS, uniform retry budgets, and request metrics — else you reintroduce language skew.
Envoy vs Linkerd-proxy?
Answer
Envoy is a general programmable L7 proxy. Linkerd-proxy is a slim mesh-focused datapath with strong mTLS defaults. Pick from ops surface and feature depth, not brand loyalty.
Can nginx be a sidecar?
Answer
Yes — common for simple reverse proxy and auth. Meshes often standardize on Envoy or Linkerd-proxy for xDS/mTLS automation.
Hot restart?
Answer
Reload config and drain listeners without dropping all connections. Envoy is known for this; nginx reload is a different mechanism. Name the goal, not the flag.
Passive vs active health?
Answer
Outlier is passive from real traffic. Active probes are health checks — health-check lesson. Use both; do not conflate.
Why observability in the proxy?
Answer
Golden signals without instrumenting every language. You still need app spans for business logic.
Why does gRPC care?
Answer
gRPC needs an HTTP/2-aware L7 proxy. A raw L4 NLB will not give you route-level retries or per-method timeouts.
Does filter order matter?
Answer
Yes. Auth before route. Decode before rate-limit on body. Wrong order is a production incident, not a style choice.
TCP passthrough?
Answer
The proxy may only wrap mTLS and emit connection metrics. Retries are connection-level. A heavy L7 sidecar is usually the wrong tax.
WebSockets and long streams?
Answer
Retries mid-stream are unsafe. Drain and idle timeouts dominate. Affinity needs, if any, live in sticky sessions — do not recap Maglev.
Where does config come from?
Answer
Control plane discovery — xDS-shaped APIs, Consul templates, or a managed mesh. Next lesson: control plane.
Is this an Envoy tutorial?
Answer
No. Envoy is one dataplane example alongside nginx and Linkerd-proxy. Interviews score the role, not the envoy.yaml trivia.