How Rate Limiter works

Before you start

• Requirements

  Each worked system starts from a scoped list of verbs and numbers.

  Open lesson

• Caching

  The read path sits behind cache-aside.

  Open lesson

Why it matters

An API key may be allowed 100 calls per minute, but real callers do not arrive at perfectly even intervals. The service should accept a legitimate short burst, reject sustained excess with a useful retry time, and keep one caller from consuming the expensive handler. Each decision must take one atomic round trip to a shared counter store, never a query to the primary database.

Each API key may call your service 100 times a minute. Over the limit, the caller receives 429 and a hint about when to retry.

Input

requests = [0, 0.1, 0.2, 0.3, 1.5, 1.6], capacity = 3, rate = 1

Output

[True, True, True, False, True, False]

A fixed minute can allow nearly 200 calls across its boundary. For the 100-per-minute API key, use a token bucket whose capacity bounds the burst and whose elapsed-time refill bounds the average, returning 429 with Retry-After when empty.

Scale the example to a capacity of 3 and a refill of 1 token per second. Requests at 0, 0.1, and 0.2 spend the initial tokens; 0.3 receives 429. By 1.5, elapsed time has restored enough for one request, and Retry-After can report the remaining fraction for a denied caller.

No timer updates idle callers. The atomic store operation reads tokens and last_seen, adds elapsed time times the refill rate up to capacity, then spends one if available. For the production API key, choose capacity and refill to represent 100 per minute. Fail-open or fail-closed remains an explicit product decision.