LLM Gateway Rate Limiter
Studies in this cluster, in series order. Each one keeps its own URL.
Low-level design
OOP and SOLID class design for machine-coding LLD: interfaces, invariants, in-memory state, and concurrency you can implement and test.
LLM Gateway Rate Limiter
3 studies- 1.Rate Limiter for an LLM Gateway - LLD Spec & ConceptsMachine-coding LLD for an in-process LLM gateway limiter: a per-tenant token bucket, an exact sliding-window log, and a composed decision with remaining and retry_after_ms.
- 2.Rate Limiter - Token Bucket + Sliding Window Solution & TestsRunnable TokenBucket, SlidingWindowLog, and GatewayLimiter, with a fake clock and an 80-thread burst flood.
- 3.Rate Limiter - Concurrency, Retry-After & CompositionWhy one RLock around the composed check still counts an RPM event when the burst denies, how retry_after_ms is computed, and how this round maps onto Redis Lua.