Caching
Part 8 of 8 · Redis cacheDistributed Caching Consistency — Local vs Shared Cache & Near-Cache
A near-cache (L1 in each pod) on top of a shared L2 (Redis) removes the network hop for hot keys, but every write must now reach every pod. Invalidate L2, fan out a versioned message to every L1, keep L1 TTL short, and flush L1 when a subscriber reconnects.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
Where does the hot key live?
Prefer
Near-cache with a versioned fan-out
L1 removes the hop for hits. A write deletes L2, publishes the key and version, and each pod drops older L1 copies. A short L1 TTL and a reconnect flush bound a lost message.
- Versions travel with the message and with the fill.
- An L1 hit does not round-trip to L2 to check freshness.
- Pub/sub does not queue for a disconnected subscriber.
Alternative
Long L1 TTL, or invalidate Redis only
Pod B keeps serving v1 after L2 already has v2. Sticky sessions do not clear the other pods.
- Users see different answers on different pods.
- A global lock on every L1 hit removes the point of the near-cache.
- Replica reads and failover can also serve or drop a write Redis already acknowledged.
Write, then make every pod drop the old L1
The failure branch is a dropped message or a pod that is reconnecting.
- 1
Commit v2
The writer commits the new row to the database. - 2
Delete the L2 key
Shared Redis no longer holds the previous blob. - 3
Publish key and version
The bus carries the key and v2. Redis pub/sub is fire-and-forget. - 4
Each pod drops older L1 copies
A local entry older than v2 is removed. The pod remembers the version it heard. - 5
Next read misses L1
The following request does not reuse the deleted local copy. - 6
Read L2, or the database
L2 miss falls through to the source of truth. - 7
Fill upward
Write L2, then L1, with the version and a short L1 TTL.
Overview
At scale you stack an L1 local cache (in-process or node-local) on an L2 shared cache (a Redis or Memcached cluster). Hits get faster, and consistency gets harder: invalidations must reach every pod, replicas lag, and a pod's L1 can keep serving an old value after L2 is fixed. This page covers the topologies, the consistency options, and the patterns that work: broadcast invalidation, versions, and a short L1 TTL. It builds on cache invalidation.
A CDN edge plus an origin cache is the HTTP form of the same hierarchy. See CDN cache hierarchy.
Comparative: topologies
| Topology | Latency | Coherence | Blast radius | Best when |
|---|---|---|---|---|
| Shared only (Redis) | One network round trip | One logical copy | Redis outage means a miss storm to the database | Medium QPS, simple operations |
| Local only | Sub-millisecond | None across pods; each pod drifts | One process | A single instance, or data that may differ per pod |
| Near-cache L1+L2 | Best on hits | Must invalidate both tiers | L1 stays stale after L2 is fixed | Ultra-hot keys, read-heavy traffic |
| CDN edge plus origin | Global | Purge APIs and TTLs | Regional staleness | Public static or semi-static content |
What fails if you choose wrong
- Long L1 TTL without an invalidation bus: users get different answers on different pods.
- Invalidating Redis only: L1 keeps the old value until its TTL.
- Treating Redis Cluster as strongly consistent: replication is asynchronous, so replica reads can be stale, and a failover can lose recently acknowledged writes.
WAITreduces that risk and does not make Redis linearizable. - A global lock or an L2 round trip on every L1 hit: you have removed the point of the near-cache.
Consistency options
- Best-effort TTL. Eventual, and the simplest.
- Invalidate on write.
DELin L2 plus a pub/sub message that drops L1 everywhere. - Versioned values. Messages and fills carry a version. L1 refuses to keep a copy older than the newest version it has heard about.
- Redis client-side caching (
CLIENT TRACKING, Redis 6+). The server tracks which keys a client read and pushes invalidation messages, so you do not hand-roll the bus. The same rule applies: on a lost connection, flush the local cache. - Leases or read repair. Rare for caches. That is database and consensus territory.
Local vs shared tradeoffs
Shared cache: one copy, one network hop, centralized memory and eviction. See eviction.
Local cache: no hop, memory multiplied by pod count, and coherence is your problem.
Near-cache: cap L1 size aggressively and keep L1 TTL in seconds even when L2 TTL is minutes. Subscribe to invalidations for the hot entities you cache locally.
Patterns that work
- Write path: commit the database,
DEL(orSETwith a version) in L2, publish(key, version), optional delayed secondDEL. - Read path: L1, then L2, then the database. Fill upward with the version.
- Reconnect: Redis pub/sub is fire-and-forget and does not queue for a disconnected subscriber, so flush L1 when the subscription reconnects.
- Hot keys: soft TTL plus single-flight at each layer. See single-flight.
- Sticky sessions do not replace invalidation. The other pods still hold the old value, and a load-balancer change sends users there.
Flow
- 1
Step 1 Writer commits v2 to DB
- nextStep 2 DEL key in L2 Redis
- 2
Step 2 DEL key in L2 Redis
- nextStep 3 Publish invalidate key v2
- 3
Step 3 Publish invalidate key v2
- nextStep 4 Each pod drops L1 copies older than v2
- Failure path - message dropped or pod reconnectingPod serves stale v1 from L1
- 4
Step 4 Each pod drops L1 copies older than v2
- nextStep 5 Next read misses L1
- 5
Step 5 Next read misses L1
- nextStep 6 Read L2, on miss load DB
- 6
Step 6 Read L2, on miss load DB
- nextStep 7 Fill L2 then L1 with version, short L1 TTL
- 7
Step 7 Fill L2 then L1 with version, short L1 TTL
- 8
Pod serves stale v1 from L1
- nextBounded by L1 TTL, flush whole L1 on reconnect
- 9
Bounded by L1 TTL, flush whole L1 on reconnect
- nextStep 5 Next read misses L1
Lesson map
Distributed Caching Consistency — Local vs Shared Cache & Near-Cache
A near-cache (L1 in each pod) on top of a shared L2 (Redis) removes the network hop for hot keys, but every write must now reach every pod. Invalidate L2, fan out a versioned message to every L1, keep L1 TTL short, and flush L1 when a subscriber reconnects.
Architecture. Step 1 Writer commits v2 to DB Ready. Step 2 DEL key in L2 Redis Ready. Step 3 Publish invalidate key v2 Ready. Step 4 Each pod drops L1 copies older than v2 Ready. Step 5 Next read misses L1 Ready. Step 6 Read L2, on miss load DB Ready. Step 7 Fill L2 then L1 with version, short L1 TTL Ready. Pod serves stale v1 from L1 Ready. Bounded by L1 TTL, flush whole L1 on reconnect Ready
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB W1["Step 1 Writer commits v2 to DB Ready"] W2["Step 2 DEL key in L2 Redis Ready"] W3["Step 3 Publish invalidate key v2 Ready"] W4["Step 4 Each pod drops L1 copies older than v2 Ready"] R5["Step 5 Next read misses L1 Ready"] R6["Step 6 Read L2, on miss load DB Ready"] R7["Step 7 Fill L2 then L1 with version, short L1 TTL Ready"] F1["Pod serves stale v1 from L1 Ready"] F2["Bounded by L1 TTL, flush whole L1 on reconnect Ready"] W1 -->|continues| W2 W2 -->|continues| W3 W3 -->|continues| W4 W4 -->|continues| R5 R5 -->|continues| R6 R6 -->|continues| R7 W3 -->|Failure path - message dropped or pod reconnecting| F1 F1 -->|continues| F2 F2 -->|continues| R5
Multi-region
Active-active caches without a global invalidation story give you per-region truth with asynchronous replication. Write the staleness SLA down. Cross-region write-through is usually the wrong shape for user-facing caches. Prefer region-local caches with origin-region database affinity, or CRDTs for special cases. Write-through inside one region is still only local: write-through vs write-behind.
Sandbox: two pods with versioned L1 invalidation (Python)
Runnable with python3, no dependencies. It shows the happy path, a dropped message bounded by L1 TTL, and the reconnect flush.
Press Run. Snippets must be self-contained — no network, files, or native modules.
Version gate without an L2 round trip (TypeScript)
Runnable with npx tsx, no dependencies. An L1 hit returns the local value. The version gate applies when filling, not on every hit.
Press Run. Snippets must be self-contained — no network, files, or native modules.
Interview Q&A
Why a near-cache if Redis is fast?
Answer
It removes the network hop and serialization for ultra-hot keys and raises the QPS ceiling. Redis stays as L2. You pay in coherence work.
How do you invalidate L1 across pods?
Answer
Pub/sub (Redis, Kafka, NATS) or Redis client-side caching, with versions on messages, a short L1 TTL as the backstop, and a full L1 flush on reconnect.
Is Redis always consistent?
Answer
A single primary serves one-key operations atomically, and replication is asynchronous. Replica reads can be stale, and failover can drop acknowledged writes. Name that window.
L1 TTL vs L2 TTL?
Answer
L1 is much shorter, seconds versus minutes, because only L1 staleness is invisible to the rest of the system.
Multi-region cache?
Answer
Regional L2 plus a documented staleness SLA. Global invalidation is slow and hard. Write-through is not cross-region linearizability.
What metric shows coherence bugs?
Answer
Cross-pod read disagreement probes and version-mismatch counters. Hit rate alone will not show pod B serving v1.
How does this relate to stampedes?
Answer
An L1 miss storm still hits L2 and the database. Apply single-flight at each layer.
Are sticky sessions a coherence strategy?
Answer
No. Stickiness reduces how often a user changes pods. It does not invalidate the other pods.
Where does stale-while-revalidate fit?
Answer
It is the HTTP edge form of a soft TTL: the same stale window at a different hop.
Pitfalls
Draw pods A and B, each with L1, sharing L2. A writes v2. Show the cache contents if only L2 is deleted, if the message reaches B, if the message is dropped with L1 TTL 30 seconds versus 2 seconds, and if B reconnects. State the staleness SLO you would put in the design doc.