CDNs, Cache Hierarchy & Origin Shielding

1. 1

   every POP miss → origin

   Flat edge, no shield

   A viral object expiry worldwide looks like a DDoS. Collapsing at one node does not collapse across POPs.

2. 2

   Winner: edge → mid-tier → shield → origin

   Each tier collapses a different fan-in. The shield is the topology choke point; in-process coalescing is the local one. Pay an extra hop on cold miss to buy origin QPS.

3. ?

   Keys, SWR, purge, TLS — the knobs that make hierarchy honest

   Vary: Cookie explodes the key. SWR hides revalidation but is not a correctness tool. Hard purge races the shield. Anycast TLS is why the first byte is fast. Those are the sibling pages — this hub stays on the path.

Overview

At senior level, interviewers probe whether you can design for hit ratio and correctness under purge, cookie variance, and origin outages. App-level cache-aside (Redis) is one loop; this lesson closes the loop at edge networking.

By the end you should be able to:

1. Draw client → edge POP → mid-tier / regional cache → origin shield → origin and explain what each tier collapses.
2. Choose Cache-Control / s-maxage / stale-while-revalidate / stale-if-error deliberately for browsers vs shared caches.
3. Design cache keys that avoid cookie-busting and incorrect Vary.
4. Explain purge vs soft purge, and why shielding creates purge races.
5. Sketch signed URLs and a purge API client (Python and TypeScript).
6. Answer senior questions on Anycast, TLS termination, and request collapsing.

The cache hierarchy

A POP (Point of Presence) is a physical/colo presence where the CDN runs edge servers. Users are steered to a nearby POP via DNS geo / GSLB or Anycast. Inside a POP you typically have many cache nodes; the "edge" is the user-facing cache layer.

Tiered cache means lower tiers ask upper tiers before the origin.

• Cloudflare Tiered Cache / Smart Tiered Cache dynamically picks an upper tier close to the origin.
• AWS CloudFront has Regional Edge Caches by default and optional Origin Shield in a chosen region.
• Fastly uses shielding (designate a shield POP).

Benefit: bandwidth efficiency + origin load reduction. Cost: extra hop latency on cold misses; concentration risk if the shield is unhealthy (vendors add shield failover).

flowchart TB
  c["Client / Browser"]
  e["Edge POP L1: TLS, WAF, cache"]
  m["Mid-tier / Regional Cache L2"]
  s["Origin Shield"]
  c -->|HTTPS Anycast or| e
  e -->|HIT| c
  e -->|MISS /| m
  m -->|HIT| e
  m -->|MISS| s
  s -->|HIT| m

Read that as: a HIT returns immediately; a MISS walks up the tree; the origin is reached only after every shared tier missed, and concurrent equivalent misses collapse at the shield.

Origin shielding

Origin Shield = force all origin-bound misses through one shared cache location closest to the origin (not closest to users).

• Collapses geographically distributed misses into one upstream path.
• Improves connection reuse (warm TCP/TLS pools to origin).
• Multi-CDN: one shield (for example CloudFront Origin Shield) can absorb other CDNs' origin traffic if you architect the pull path carefully.

Choose the shield region by lowest latency to origin, not to users. Shielding is a capacity control. Any latency win is a side effect of connection reuse and fewer origin trips.

Freshness: Cache-Control, s-maxage, SWR

HTTP caching (RFC 9111) is the contract. Browsers and CDNs do not read the same directives the same way. Soft vs hard TTL, SWR coalescing, and stale-if-error during origin 5xx: SWR & stale-if-error. Purge races with the shield: Purge & generation tokens.

Practical pattern for HTML/API that must be eventually fresh:

Cache-Control: public, max-age=60, s-maxage=3600, stale-while-revalidate=86400, stale-if-error=604800

Browsers refresh often (max-age=60); the CDN holds for an hour; soft stale covers spikes and origin blips.

Cache keys

Default key ≈ (scheme, host, path, query?) plus selected headers from Vary. Normalization, cookie-busting, and when Vary is the wrong tool: Cache keys & Vary.

Senior pitfalls:

• Including all query params fragments the cache (utm_*, session ids).
• Vary: Cookie or Vary: * → near-zero hit ratio.
• Ignoring normalized path / trailing slash / case → duplicate objects.
• Forgetting device / image format variants (Accept) when you do need variants — use controlled Vary or separate URLs.

Prefer: whitelist query params in the key; use surrogate keys / tags for purge groups.

Purge and invalidation

Shielding + purge race: edge purged first may refetch from an unpurged shield and re-cache old content. Mitigations: purge twice (Fastly documents this), generation tokens in the cache key, soft purge + careful Age handling.

Never casually hard-purge-all. That is how you buy a thundering herd.

TLS termination and Anycast

TLS. The edge terminates client TLS (short RTT handshake). Origin connections are often long-lived HTTPS (or mTLS) from shield/edge → origin. Handshake budgets, Anycast vs geo-DNS, and key custody: TLS termination & Anycast edge.

• Faster TTFB, WAF inspection, cert management at the CDN.
• Tradeoff: the CDN sees plaintext (trust / compliance). Still pull origin over TLS; that is hop encryption, not end-to-end privacy from the CDN.

Anycast. The same IP is announced from many POPs; BGP delivers packets to a topologically "close" POP.

Many CDNs use Anycast for DNS + content (Cloudflare) or mix with DNS steering.

Request collapsing (coalescing)

When N concurrent requests miss the same key at one cache node, one fetch goes upstream; waiters share the response. That prevents a local thundering herd. It complements the shield, which collapses across POPs. Single-flight, uncacheable leaders, and in-memory demos: Request collapsing / coalescing.

Caveat: a slow uncacheable leader can inflate waiters' latency — tune timeouts and do not collapse uncacheable responses forever.

Failure modes

Design for origin outage: long stale-if-error, last-known-good from edge/shield, negative-cache 5xx briefly with care, multi-origin failover for safe GETs, and do not disable collapsing during incidents. Availability can stay high while the freshness SLO slips — say that tradeoff out loud.

Worked examples

Cache-Control helpers are the boring, correct part. Cache keys and collapsing are what interviewers ask you to implement.

Press Run. Snippets must be self-contained — no network, files, or native modules.

Press Run. Snippets must be self-contained — no network, files, or native modules.

Cache-Control builders (teaching sketches)

def build_cache_control(
    *,
    max_age: int,
    s_maxage: int | None = None,
    swr: int | None = None,
    sie: int | None = None,
    public: bool = True,
    immutable: bool = False,
) -> str:
    """Browser TTL is max-age; CDN TTL is s-maxage."""
    parts = ["public" if public else "private", f"max-age={max_age}"]
    if s_maxage is not None:
        parts.append(f"s-maxage={s_maxage}")
    if swr is not None:
        parts.append(f"stale-while-revalidate={swr}")
    if sie is not None:
        parts.append(f"stale-if-error={sie}")
    if immutable:
        parts.append("immutable")
    return ", ".join(parts)

HTML_CC = build_cache_control(max_age=60, s_maxage=3600, swr=86_400, sie=604_800)
ASSET_CC = build_cache_control(max_age=31_536_000, s_maxage=31_536_000, immutable=True)

export function buildCacheControl(opts: {
  maxAge: number;
  sMaxAge?: number;
  swr?: number;
  sie?: number;
  isPublic?: boolean;
  immutable?: boolean;
}): string {
  const parts: string[] = [opts.isPublic === false ? "private" : "public"];
  parts.push(`max-age=${opts.maxAge}`);
  if (opts.sMaxAge !== undefined) parts.push(`s-maxage=${opts.sMaxAge}`);
  if (opts.swr !== undefined) parts.push(`stale-while-revalidate=${opts.swr}`);
  if (opts.sie !== undefined) parts.push(`stale-if-error=${opts.sie}`);
  if (opts.immutable) parts.push("immutable");
  return parts.join(", ");
}

Signed URLs (not runnable here — HMAC + URL parsing)

Edge verifies exp + sig before serving a private object. Sign the canonical URL so extra query params cannot forge access. Bind expiry + path; compare in constant time.

import hashlib
import hmac
from urllib.parse import parse_qs, urlencode, urlsplit, urlunsplit

def sign_url(base_url: str, *, expires_at: int, secret: bytes) -> str:
    message = f"{base_url}|exp={expires_at}".encode()
    sig = hmac.new(secret, message, hashlib.sha256).hexdigest()
    sep = "&" if "?" in base_url else "?"
    return f"{base_url}{sep}exp={expires_at}&sig={sig}"

def verify_signed_url(url: str, *, secret: bytes, now: int, skew_seconds: int = 30) -> bool:
    parts = urlsplit(url)
    q = parse_qs(parts.query)
    try:
        exp = int(q["exp"][0])
        sig = q["sig"][0]
    except (KeyError, ValueError, IndexError):
        return False
    if now > exp + skew_seconds:
        return False
    filtered = [(k, v[0]) for k, v in q.items() if k not in {"exp", "sig"}]
    base = urlunsplit((parts.scheme, parts.netloc, parts.path, urlencode(filtered), ""))
    expected = hmac.new(secret, f"{base}|exp={exp}".encode(), hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, sig)

import { createHmac, timingSafeEqual } from "crypto";

export function signUrl(baseUrl: string, expiresAt: number, secret: string): string {
  const message = `${baseUrl}|exp=${expiresAt}`;
  const sig = createHmac("sha256", secret).update(message).digest("hex");
  const sep = baseUrl.includes("?") ? "&" : "?";
  return `${baseUrl}${sep}exp=${expiresAt}&sig=${sig}`;
}

export function verifySignedUrl(
  url: string,
  secret: string,
  now = Math.floor(Date.now() / 1000),
  skewSeconds = 30,
): boolean {
  const u = new URL(url);
  const exp = Number(u.searchParams.get("exp"));
  const sig = u.searchParams.get("sig");
  if (!exp || !sig) return false;
  if (now > exp + skewSeconds) return false;
  u.searchParams.delete("exp");
  u.searchParams.delete("sig");
  const base = u.toString().replace(/\?$/, "");
  const expected = createHmac("sha256", secret).update(`${base}|exp=${exp}`).digest("hex");
  const a = Buffer.from(expected);
  const b = Buffer.from(sig);
  return a.length === b.length && timingSafeEqual(a, b);
}

Signed URLs vs signed cookies: URLs protect individual objects (downloads, HLS segments). Cookies cover path prefixes (a private tree) with one credential. Prefer URLs for one-off assets; cookies for browsing a private section.

Purge API client (not runnable here — network)

Soft purge after CMS publish so SWR can refill without a hard-MISS spike. With shielding, consider a second purge after the shield has taken the first.

import json
import urllib.request
from typing import Iterable

class CDNPurgeClient:
    def __init__(self, *, api_base: str, api_token: str, zone_id: str):
        self.api_base = api_base.rstrip("/")
        self.api_token = api_token
        self.zone_id = zone_id

    def _request(self, method: str, path: str, body: dict) -> dict:
        data = json.dumps(body).encode()
        req = urllib.request.Request(
            f"{self.api_base}{path}",
            data=data,
            method=method,
            headers={
                "Authorization": f"Bearer {self.api_token}",
                "Content-Type": "application/json",
            },
        )
        with urllib.request.urlopen(req, timeout=30) as resp:
            return json.loads(resp.read().decode())

    def purge_urls(self, urls: Iterable[str], *, soft: bool = False) -> dict:
        payload: dict = {"files": list(urls)}
        if soft:
            payload["soft"] = True  # vendor-specific flag / Soft-Purge header elsewhere
        return self._request("POST", f"/zones/{self.zone_id}/purge_cache", payload)

    def purge_prefixes(self, prefixes: Iterable[str]) -> dict:
        # Prefix purge is directory-scoped — no query strings in prefixes.
        return self._request(
            "POST",
            f"/zones/{self.zone_id}/purge_cache",
            {"prefixes": list(prefixes)},
        )

export class CDNPurgeClient {
  constructor(
    private apiBase: string,
    private apiToken: string,
    private zoneId: string,
  ) {}

  private async request(path: string, body: unknown): Promise<unknown> {
    const res = await fetch(`${this.apiBase}${path}`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${this.apiToken}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify(body),
    });
    if (!res.ok) throw new Error(`purge failed: ${res.status}`);
    return res.json();
  }

  purgeUrls(urls: string[], soft = false) {
    return this.request(`/zones/${this.zoneId}/purge_cache`, { files: urls, soft });
  }

  purgePrefixes(prefixes: string[]) {
    return this.request(`/zones/${this.zoneId}/purge_cache`, { prefixes });
  }
}

Interview Q&A

Pitfalls

Cheat sheet

Go Deeper

Docs

• Cloudflare Tiered Cache
• Cloudflare purge by prefix
• Fastly freshness / HTTP caching semantics
• Fastly stale / SWR
• Fastly purging (including shielding races)
• AWS CloudFront Origin Shield
• RFC 9111 HTTP Caching

Talks

• What Is A CDN? How Does It Work?
• Demystifying CDNs
• Why Your Web Service Needs an Origin Shield (Varnish)
• CDN Architecture: Origin vs Edge
• Cloudflare resilient systems / Anycast

Extra

• Using CloudFront Origin Shield in a multi-CDN deployment