How would you design a rate limiter for an API?
System designMid-level
In shortClarify who is limited and how strictly, pick an algorithm (usually a token bucket or sliding window counter), keep counters in a shared store like Redis updated atomically, return 429 with Retry-After, and decide what happens when the limiter fails.
Model answer
- Clarify: who is limited (user, API key, IP, tenant), which endpoints, the limits, whether bursts are OK, the scale, and the latency budget.
- Algorithm: a token bucket (bursts up to the bucket size, then a steady rate) or a sliding window counter (two counters per key, close to exact). Mention the fixed window’s 2× burst at boundaries and the sliding log’s memory cost.
- Where: in the API gateway, before traffic reaches services, with shared counters in Redis so limits hold across all instances. Update them atomically with a Lua script, with expiry in the same step.
- Response:
429 Too Many RequestswithRetry-AfterandRateLimit-*headers; clients back off with jitter. - Failure: short timeouts to Redis; fail open for most endpoints, fail closed for abuse-sensitive ones like login.
At scale, trade exactness for latency: gateways lease tokens in batches for hot keys, and regions enforce shares of a global limit instead of coordinating on every request.
What they're looking for
Clarifying questions, at least two algorithms with trade-offs, shared atomic counters across instances, the HTTP contract, and failure policy. Senior answers add scale trade-offs (batching, regions) and per-endpoint fail-open/closed decisions.
Common mistakes
- Per-server in-memory counters behind a load balancer.
- Separate INCR and EXPIRE calls, or ignoring atomicity entirely.
- Forgetting what the client sees and how it should back off.