Networking
Part 2 of 6 · Sidecar & Service MeshSidecar Mechanics — Transparent Proxy, iptables/eBPF Capture & Container Lifecycle
A sidecar only works if app traffic actually hits it. Transparent capture (redirect without app config) and pod lifecycle (init, ready, drain) are where interviews get concrete. Envoy, nginx, and Linkerd-proxy all need a capture story and a start/stop story.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
How the packet gets into the proxy
Prefer
Transparent capture the app does not configure
The workload still dials service DNS or a VIP. The platform steals the socket into the sidecar. That is the interview promise — prove it with conntrack, bpftool, or proxy admin stats.
- iptables/nftables: mature, debuggable, per-packet cost and CNI fragility.
- eBPF: faster path and ambient modes; kernel version and harder failure modes.
- Drain is the other half of the story: stop new inbound, wait in-flight, then kill the app.
Alternative
Hope the app set HTTP_PROXY, or SIGKILL both containers
Explicit env works until one language forgets. Naive kill drops in-flight work. Redirect without UID exclusion loops until the node falls over.
- Explicit mode is not zero-app-change.
- App-before-redirect races leak plaintext past the mesh.
- Probes through mTLS fail closed and look like a dead pod.
Lifecycle the interviewer wants named
Capture without drain is half a sidecar. Reverse of naive SIGKILL.
- 1
Init installs rules
Redirect or wait for CNI. Native sidecar containers change this on recent Kubernetes. - 2
Proxy listens
Inbound and outbound ports; admin / readiness. Depth: data-plane proxies. - 3
App starts
Race if it binds before redirect is ready — first requests escape. - 4
Both ready
Some meshes inject readiness gates so kube does not send traffic early. - 5
Drain on SIGTERM
Stop new inbound, wait in-flight, then the app exits. Mis-order drops work.
Overview
A sidecar only works if app traffic actually hits it. Transparent capture (redirect without app config) and pod lifecycle (init, ready, drain) are where interviews get concrete. This page is proxy-agnostic: Envoy, nginx, or Linkerd-proxy all need a capture story and a start/stop story.
You should be able to:
- Pick capture mode from "can the app change?" and "do we need a kernel path?"
- Name the redirect-loop fix (exclude proxy UID, or mark packets).
- Align probes with health checks so kube does not mTLS its own liveness.
Transparent capture models
| Model | App change? | Ops surface | Typical miss |
|---|---|---|---|
| Explicit proxy | Yes — localhost or HTTP_PROXY | App config | One language forgets; mixed fleet |
| iptables / nftables REDIRECT or TPROXY | No | Init container, NET_ADMIN | CNI fights, loops, UDP, IPv6 holes |
| eBPF / sockmap / cgroup | No | Node agent or CNI | Kernel features; harder to reason about |
| CNI / netns tricks | No | Platform-specific | Rare outside a product datapath |
Decisions
- ?
1 Zero app change?
- no2 Explicit localhost
- yes3 Kernel or node path?
- 2
2 Explicit localhost
- ?
3 Kernel or node path?
- no4 iptables TPROXY
- yes5 eBPF or ambient
- 4
4 iptables TPROXY
- 5
5 eBPF or ambient
Lesson map
Sidecar Mechanics — Transparent Proxy, iptables/eBPF Capture & Container
A sidecar only works if app traffic actually hits it. Transparent capture (redirect without app config) and pod lifecycle (init, ready, drain) are where interviews get concrete. Envoy, nginx, and Linkerd-proxy all need a capture story and a start/stop story.
Architecture. Architecture
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB q["1 Zero app change?"] ex["2 Explicit localhost"] k["3 Kernel or node path?"] ip["4 iptables TPROXY"] q -->|no| ex q -->|yes| k k -->|no| ip
iptables vs eBPF
- iptables: mature, debuggable with conntrack; fragile with custom CNIs; per-packet overhead; races on rule install.
- eBPF: faster path, richer hooks; harder to reason about; kernel/version requirements; different failure modes.
- Both can break hostNetwork, hairpin, or UDP-heavy workloads if misapplied.
REDIRECT vs TPROXY: REDIRECT rewrites the destination to a local port. TPROXY preserves the original destination so the proxy can route on it. Original-destination routing is why TPROXY shows up in interviews.
Capture path
The app connects to a service VIP. Redirect steals to the local proxy. The proxy applies mTLS, retries, and metrics, then talks to a peer sidecar or gateway.
Flow
- 1
1 App dials VIP
- next2 iptables or eBPF
- 2
2 iptables or eBPF
- next3 Local sidecar port
- 3
3 Local sidecar port
- mTLS + retries4 Peer sidecar
- 4
4 Peer sidecar
- next5 Peer app
- 5
5 Peer app
Deep dive · Platform notes (Kubernetes-shaped)
Native sidecar containers (sidecars as restartable init) change drain and probe semantics — call them out if the interviewer is on recent Kubernetes.
CNI conflict checklist: Calico, Cilium, and cloud CNIs may need documented redirect modes; never assume iptables works unchanged.
IPv6 dual-stack: redirect rules must cover both families or you get mystery bypass.
StartupProbe on the app should not fire until capture is ready — otherwise first requests escape the proxy.
Drain order
Naive SIGKILL is the reverse of what you want.
Flow
- 1
1 SIGTERM
- next2 Stop new inbound
- 2
2 Stop new inbound
- next3 Wait in-flight
- 3
3 Wait in-flight
- next4 Then stop the app
- 4
4 Then stop the app
- next5 Pod stopped
- 5
5 Pod stopped
Connection draining at an LB is the cousin lesson: health checks, slow start, drain. Do not recap unhealthy_threshold here.
Sandbox: capture cost model (Python)
Toy scoring for interviews — not a benchmark. Explicit mode fails if you required zero app change.
Press Run. Snippets must be self-contained — no network, files, or native modules.
Same idea: lifecycle (TypeScript)
Press Run. Snippets must be self-contained — no network, files, or native modules.
Common failure modes
- Redirect loops — traffic to the sidecar is redirected again. Mark packets or exclude the proxy UID.
- DNS / health-check bypass — probes that should not go through the mTLS path. See health checks.
- Privilege —
NET_ADMINfor iptables; eBPF needsCAP_BPF/ kernel features. - Job / CronJob short lives — sidecar overhead dominates for tiny tasks. Skip injection.
- hostNetwork pods — capture and port conflicts often break. Anti-pattern for mesh injection.
Identity is applied after capture. Depth: s2s mTLS / SPIFFE.
Pitfalls
Pod with app UID 1000 and proxy UID 1337. Outbound SYN to payments:8080. Where does conntrack send it? Now the proxy dials the real endpoint — why must that packet skip the same rule? Draw SIGTERM order for a 30s in-flight gRPC stream.
Interview Q&A
Why transparent proxy?
Answer
Apps keep calling service DNS or a VIP. The platform steals packets into the sidecar so every language gets policy without setting a proxy env.
iptables REDIRECT vs TPROXY?
Answer
REDIRECT changes the destination to local. TPROXY preserves the original destination for the proxy to read — that matters for original-destination routing.
Why exclude the proxy UID?
Answer
Prevent redirect loops when the sidecar itself dials out. Marked packets or UID match-not are the usual fixes.
eBPF advantage?
Answer
Potentially lower overhead and node-level ambient modes without per-pod iptables. Cost: kernel features and a different debug story.
Drain order?
Answer
Stop new inbound → wait in-flight → then kill the app. The reverse of naive SIGKILL. Cousin: LB connection draining.
Does the app need to know?
Answer
Not with transparent capture. Yes with explicit localhost proxy env.
How do health checks interact?
Answer
Platform probes may need to skip mesh mTLS. Active LB probes and kube probes are different paths — see the health-check lesson.
hostNetwork pods?
Answer
Capture and port conflicts often break. Call this out as an anti-pattern for mesh injection.
Short-lived Jobs?
Answer
Sidecar start and CPU dominate a 2-second CronJob. Leave batch namespaces uninjected.
IPv6 dual-stack miss?
Answer
Redirect rules that cover only IPv4 leak IPv6. Mystery bypass in interviews.
How do you prove capture works?
Answer
conntrack or nft list, bpftool, and proxy admin stats showing inbound/outbound. Transparent is a promise until you show packets.
Native Kubernetes sidecars?
Answer
Restartable init/sidecar containers change probe and drain semantics. Mention them if the interviewer is on recent Kubernetes — do not handwave initContainers from 2018.