Cache TTL Design & Jitter

Overview

The cache-aside hub told you to put a TTL on every fill. This lesson is how to pick that number, why a constant is a footgun, and what to do when one key is too hot for expiry.

By the end of this lesson you should be able to:

• Say out loud: TTL is a staleness bound, not a freshness guarantee
• Draw an expiry histogram with identical TTLs vs jittered TTLs
• Pick base TTL from "how wrong can this be, for how long" plus reload cost
• Explain jitter vs single-flight vs XFetch / stale-while-revalidate — they are layers, not rivals
• Keep negative-cache TTLs much shorter than positive ones

TTL is a staleness bound

Redis EXPIRE / SET … EX makes the key vanish. The next GET is a miss. Until that moment, you might be serving a value that the primary has already replaced — if some writer forgot to DEL.

That is the contract:

• Invalidate-on-write (or write-through SET) is how you try to be fresh.
• TTL is how long you are willing to be wrong when that path fails.
• No TTL is not EX 0. SET with EX 0 is rejected (invalid expire time) and EXPIRE key 0 deletes the key at once. A key with no TTL at all (TTL returns -1) never expires - memory risk; use an eviction policy consciously. See eviction.

Too-short TTL: constant reloads, Redis becomes a noisy proxy, the DB stays hot, you paid for a cache and kept the original bottleneck.

Too-long TTL: a bug or a missed invalidate becomes a prolonged incident. "We shipped the price change at 10:00 and customers saw it at noon" is a TTL number you cannot defend.

Expiry cliffs

Warm a catalog at deploy: 50,000 keys, all EX 300, all SET within a few seconds. Five minutes later they expire together. Every pod misses. Even with single-flight you now have 50,000 distinct keys to reload — the mutex helps per key, not across the keyspace.

Identical TTLs are a synchronized herd. Jitter turns one spike into a ridge.

flowchart TB
  S["Step 1 SET key EX base plus random 0 to J Ready"]
  E["Step 2 Expiries spread over the J window Ready"]
  G["Step 3 GET - hit or miss? Ready"]
  H["Step 4 Inside the soft-TTL or XFetch window? Ready"]
  R["Step 5a Return the value Ready"]
  B["Step 5b Return the value and refresh in the background Ready"]
  L["Step 6 Single-flight load, SET with a new jittered TTL Ready"]
  F["Failure path - synchronized cliff, DB QPS spike Ready"]
  S -->|continues| E
  E -->|continues| G
  G -->|hit| H
  H -->|no| R
  H -->|yes| B
  G -->|miss| L
  S -->|same TTL on a batch warm| F

Jitter recipe

ttl_seconds = base + uniform(0, J)

Typical J is 10–20% of base (30s on a 300s base). Wider than the base and you no longer have a bound — some keys live 2 * base, which you did not agree to with the product.

Apply jitter on every fill, including the SET after a single-flight load. Forgetting it there re-creates the cliff the mutex just survived.

Soft TTL, hard TTL, XFetch

Jitter spreads many keys. One viral key still expires at one instant. Layer more:

Hard TTL is Redis EXPIRE. The key is gone. Next read is a miss.

Soft TTL is an application timestamp you store next to the value (or a separate logical expiry). Past soft TTL you still return the blob and trigger a refresh. Past hard TTL Redis dropped it and you must load. This is stale-while-revalidate for application caches.

XFetch (Vattani, Grosvenor, Rodriguez, VLDB 2015) — Optimal Probabilistic Cache Stampede Prevention. Refresh probabilistically before expiry; the chance grows with recompute time (delta) and a tuning factor β. Store (value, delta, logical_expiry) with a physical Redis TTL slightly longer than logical expiry. On each hit, refresh early if:

now - delta * beta * ln(random()) >= expiry

• delta = measured recompute cost
• beta ≈ 1.0 (higher = earlier refresh)
• random() in (0, 1) so ln is negative: that product is a random lead time
• Hot keys are sampled more often, so they refresh earlier — no global lock
• Occasional redundant refreshes are OK
• Combine with a mutex for hard misses (value already gone)

Clock skew: prefer Redis's own TTL (PTTL, EX) over app clocks when you can. If you store an absolute expires_at, use Redis TIME (or the same clock the SET used), not each pod's wall clock. A pod 2 seconds fast will XFetch late or early as a herd.

Per-key classes

Do not run one TTL for the whole cache.

Reload cost matters. A 200ms join that is acceptable at 10 QPS is not acceptable at 10k QPS when a cliff hits. Expensive keys get longer TTL or XFetch, not "make TTL 5 seconds so it feels fresh." Freshness comes from DEL on write.

What to measure

Hit ratio alone hides cliffs. Watch:

• Miss rate and miss latency over time (look for a comb at base seconds)
• DB QPS correlated with TTL boundaries
• Lock acquire rate and lock wait timeouts (stampede the mutex did not fully hide)
• Rebuild duration delta (feeds XFetch)
• Stale-serve count (soft TTL is working)
• Evicted keys vs expired keys (eviction is not TTL; see the last lesson)

Alert when miss rate or DB QPS spikes at TTL multiples after a warm or a deploy.

Working sketches

# Sketch of the helpers. Playground below runs them on 200 keys.
import random, time

def ttl_with_jitter(base_seconds: int, jitter: int = 30) -> int:
    if base_seconds < 1:
        raise ValueError("base_seconds must be >= 1")
    return base_seconds + random.randint(0, max(0, jitter))

def should_soft_refresh(expires_at: float, now: float | None = None, window: float = 30.0) -> bool:
    now = time.time() if now is None else now
    return expires_at - now <= window

export function ttlWithJitter(baseSeconds: number, jitter = 30): number {
  if (baseSeconds < 1) throw new Error("baseSeconds must be >= 1");
  return baseSeconds + Math.floor(Math.random() * (Math.max(0, jitter) + 1));
}

export function shouldSoftRefresh(expiresAt: number, now = Date.now() / 1000, window = 30): boolean {
  return expiresAt - now <= window;
}

In-memory histogram (run this)

200 keys, identical TTL vs jittered TTL. Print the expiry-second histogram. Then sample XFetch near vs far from expiry. No Redis, no network.

Press Run. Snippets must be self-contained — no network, files, or native modules.

You should see one bucket of 200 at t+300 without jitter, and ~31 buckets with a max around 10 when J = 30. XFetch fires often at now = 290 and almost never at now = 10 for expiry = 300.

Soft TTL plus try-lock (run this)

Store {value, soft_exp, hard_exp}. After soft expiry, losers serve stale while one winner reloads. A simplified early-refresh coin-flip (β / (ttl_remaining + β)) is a teaching model — production XFetch uses the log formula above.

Press Run. Snippets must be self-contained — no network, files, or native modules.

Interview Q&A

Pitfalls

Go Deeper

Official docs and the paper

• Redis EXPIRE
• Redis SET
• Vattani et al., Optimal Probabilistic Cache Stampede Prevention (VLDB 2015)
• Preventing cache stampede with Redis and XFetch — Jim Nelson

In this cluster

• Redis cache-aside, invalidation, stampede
• Next: Single-flight locking for cache fills
• HTTP cousin: SWR / stale-if-error