Networking
Part 6 of 6 · Sidecar & Service MeshMesh vs Library vs Gateway — Latency Tax, Blast Radius & Migration
Cross-cutting policy has three classic homes: in-process library, per-pod sidecar/mesh, and edge gateway. Ambient/eBPF is a fourth. Each shifts latency, failure blast radius, and who upgrades what. This page ties the cluster together — and repeats when not to mesh.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
Where cross-cutting concerns live
Prefer
Put policy where the pain and the blast radius match
Ingress auth and rate limits at a gateway. Uniform polyglot mTLS and retries in a mesh (or Lattice). One language plus a tight p99: a library. Sidecar density hurting bin-pack: evaluate ambient and accept node blast radius.
- Fastest path is in-process if you can standardize the language.
- Most uniform policy is mesh/sidecar or ambient across languages.
- A gateway does not replace a mesh — different directionality.
Alternative
Mesh plus library retries, or a Friday-night fleet injection
Double budgets amplify outages. Big-bang xDS without canaries takes every injected pod with a bad route. CronJobs do not want sidecars.
- Library bug → services that bumped the dep. Sidecar CVE → the injected fleet.
- Gateway outage → north-south. East-west may survive.
- Node agent failure → many pods on that node.
Migration interviewers like
Earn each stage with golden signals. Partial injection beats Friday cutover.
- 1
Gateway first
Ingress auth, TLS, WAF, rate limits. North-south only. - 2
Libraries on critical clients
Timeouts, retries, metrics on the hottest paths. - 3
Partial mesh
Inject namespaces that need mTLS or canary. Leave batch jobs alone. - 4
Full mesh only if the tax pays
Prove CPU, mem, and p99. Canary xDS. - 5
Ambient if density hurts
Accept node-level blast radius. Not a free lunch.
Overview
Architecture interviews score you on placement of cross-cutting concerns. Three classic homes: in-process library, per-pod sidecar/mesh, and edge gateway. Add ambient/eBPF as a fourth. Each shifts latency, failure blast radius, and who upgrades what.
This page ties the cluster together with migration paths — and repeats when not to mesh.
Side-by-side
| Home | Hop | Uniformity | Blast radius |
|---|---|---|---|
| Library (Resilience4j, OTel SDK, gRPC interceptors) | Tens to hundreds of µs | One language | Services that bumped the dep |
| Sidecar / mesh | Hundreds of µs to low ms (+ TLS) | All injected languages | Every injected pod on a bad push |
| Gateway (Kong, ALB, Envoy Gateway) | Edge only | North-south clients | All ingress; east-west may survive |
| Ambient / eBPF | Often less than sidecar | Fleet, node path | Node agent / kernel |
Always measure on your workload — numbers are for ranking, not quoting as law.
Decision: where does policy live?
Decisions
- ?
1 Pain is north-south?
- yes2 Edge gateway
- no2 Many languages?
- 2
2 Edge gateway
- ?
2 Many languages?
- no3 Hop fits p99?
- yes3 Sidecar tax OK?
- ?
3 Hop fits p99?
- no4 In-process library
- yes4 Library still fine
- 5
4 In-process library
- 6
4 Library still fine
- ?
3 Sidecar tax OK?
- yes4 Sidecar mesh
- no4 Ambient or Lattice
- 8
4 Sidecar mesh
- 9
4 Ambient or Lattice
Lesson map
Mesh vs Library vs Gateway — Latency Tax, Blast Radius & Migration
Cross-cutting policy has three classic homes: in-process library, per-pod sidecar/mesh, and edge gateway. Ambient/eBPF is a fourth. Each shifts latency, failure blast radius, and who upgrades what. This page ties the cluster together — and repeats when not to mesh.
Architecture. Architecture
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB dir["1 Pain is north-south?"] gw["2 Edge gateway"] lang["2 Many languages?"] hop["3 Hop fits p99?"] dir -->|yes| gw dir -->|no| lang lang -->|no| hop
Cloud buy-vs-build: App Mesh / ALB / Lattice.
Blast radius of a bad push
Library vs sidecar:
Decisions
- ?
1 What failed?
- library2 One service version
- sidecar push3 Injected fleet
- 2
2 One service version
- 3
3 Injected fleet
Gateway vs node agent:
Decisions
- ?
1 What failed?
- gateway2 Ingress clients
- node agent3 Pods on that node
- 2
2 Ingress clients
- 3
3 Pods on that node
Mitigate mesh blast with canary xDS, pod disruption budgets, and progressive rollout. Depth of how config fans out: control plane.
Latency tax (order of magnitude)
- In-process interceptor: tens to hundreds of µs
- Localhost sidecar hop: hundreds of µs to low ms (TLS adds more)
- Extra AZ / LB hop: milliseconds
Gateways and sidecars are often L7; NLB stays L4. Point at L4 vs L7 — do not recap the matrix. Long-lived affinity, if the protocol forces it, is sticky sessions.
Anti-patterns
- Mesh + duplicate library retries — retry storms. Budget in one layer.
- Gateway-only security while pods speak plaintext east-west.
- Injecting sidecars into every CronJob / short task.
- Big-bang mesh cutover Friday night without canary xDS.
- mTLS in both library and proxy with two authorities. Pick one. See s2s mTLS / SPIFFE.
Readiness must align with control-plane warmup and outlier ejection: health checks.
Sandbox: latency + blast scoring (Python)
Press Run. Snippets must be self-contained — no network, files, or native modules.
Same idea: migration stages (TypeScript)
Press Run. Snippets must be self-contained — no network, files, or native modules.
Pitfalls
A 40-service polyglot fleet, p99 budget 20ms, platform team owns pages, pain is east-west mTLS. Walk library vs sidecar vs ambient vs Lattice. Now change to one language and a 1ms budget. What flips?
Interview Q&A
Fastest path?
Answer
In-process library — if you can standardize the language.
Most uniform policy?
Answer
Mesh/sidecar (or ambient) across languages.
Does a gateway replace a mesh?
Answer
No. North-south vs east-west. Different blast radii. A gateway API may front a mesh; it is not the mesh.
Double retries?
Answer
Coordinate budgets. Prefer one layer for retries. Proxy plus SDK both retrying is how you melt a dependency.
Blast radius of a bad RDS push?
Answer
All proxies that ACKed. Canary the control plane. Depth: discovery.
When ambient?
Answer
Sidecar CPU/mem tax dominates and the team accepts node-level agents. Not automatic.
Lifecycle migration risk?
Answer
Partial injection plus traffic mirroring / dark canaries before enforce mTLS. Leave CronJobs out.
Cost center?
Answer
Engineer time plus vCPU of proxies often exceeds control-plane SaaS fees. Same warning as cloud equivalents.
Who enforces the policy you just placed?
Answer
The data plane — Envoy, nginx, Linkerd-proxy — or the library if you skipped the mesh.
Sticky sessions?
Answer
Only as an edge concern for long-lived protocols. See sticky vs stateless. Do not recap Maglev.
Health checks?
Answer
Readiness must align with warmup and outlier ejection. Health checks.
When not to mesh? (the closer)
Answer
One language, tight latency, app team pages, pain is ingress, ALB/NLB plus timeouts suffice. Say it out loud.