A miss storm

Before you start

• Caching

  Cache-aside is the read path everything here builds on.

  Open lesson

Why it matters

A popular “home” entry can expire while ten thousand requests are arriving. They all observe the same absence before any one response restores the value, turning one routine expiry into a sudden database overload. The service must keep those callers moving while the database performs one refill for “home,” not ten thousand identical reads.

A popular cache key expires, and ten thousand requests miss at once. Each would query the database.

Input

requests = ["home", "home", "about", "home", "home", "home"], cached = {"about"}

Output

1

Instead of letting all ten thousand misses load “home,” let the first request mark that key in flight and refill it once; the other 9,999 wait for that result, while a short lock TTL covers a dead loader.

The first miss creates a short-lived in-flight marker for “home” and starts the database query. Later misses find that marker and wait for its result, or serve a stale copy within an allowed grace window. When the result arrives, it fills the cache and releases every waiter.

The marker needs its own short expiry so another request can recover if the loader dies. Random TTL jitter prevents many unrelated keys from expiring together, and early refresh reduces the chance that the hottest key ever becomes absent. Waiters trade a little latency for database safety.