API Rate Limiting Patterns That Scale
Token buckets, sliding windows, and distributed rate limiting — practical patterns to protect your API without degrading the experience for legitimate users.
Every public API needs rate limiting. Without it, a single misbehaving client can saturate your database, exhaust your compute budget, or take down the entire service. But naive rate limiting — a simple counter that resets every minute — creates cliff effects and penalizes legitimate burst traffic.
Fixed Window: Simple but Flawed
The simplest approach: count requests per time window. When the counter exceeds the limit, reject.
// ❌ Fixed window — boundary burst problem
async function fixedWindowLimit(
key: string,
limit: number,
windowMs: number,
): Promise<boolean> {
const window = Math.floor(Date.now() / windowMs);
const counterKey = `ratelimit:${key}:${window}`;
const count = await redis.incr(counterKey);
if (count === 1) await redis.pexpire(counterKey, windowMs);
return count <= limit;
}
// Problem: at 11:59:59, a client sends 100 requests (under the limit).
// At 12:00:01, they send another 100. Both pass — 200 requests in 2 seconds.The boundary problem means a client can effectively double their rate limit by timing requests across window boundaries.
Sliding Window Log
Track individual request timestamps and count how many fall within the sliding window. More accurate, but uses more memory.
async function slidingWindowLog(
key: string,
limit: number,
windowMs: number,
): Promise<boolean> {
const now = Date.now();
const windowStart = now - windowMs;
const sortedSetKey = `ratelimit:${key}`;
// Remove expired entries and add current request atomically
const pipeline = redis.pipeline();
pipeline.zremrangebyscore(sortedSetKey, 0, windowStart);
pipeline.zadd(sortedSetKey, now, `${now}:${Math.random()}`);
pipeline.zcard(sortedSetKey);
pipeline.pexpire(sortedSetKey, windowMs);
const results = await pipeline.exec();
const count = results![2][1] as number;
return count <= limit;
}This eliminates the boundary problem but stores one entry per request. For high-traffic endpoints, that's a lot of Redis memory.
Token Bucket: The Best Default
Token buckets allow bursts while enforcing an average rate. Tokens are added at a steady rate; each request consumes a token. When the bucket is empty, requests are rejected.
// ✅ Token bucket — allows bursts, enforces average rate
async function tokenBucket(
key: string,
capacity: number,
refillRate: number, // tokens per second
): Promise<{ allowed: boolean; remaining: number }> {
const bucketKey = `bucket:${key}`;
const now = Date.now();
// Lua script for atomicity
const script = `
local bucket = redis.call('HMGET', KEYS[1], 'tokens', 'lastRefill')
local tokens = tonumber(bucket[1]) or tonumber(ARGV[1])
local lastRefill = tonumber(bucket[2]) or tonumber(ARGV[3])
local capacity = tonumber(ARGV[1])
local refillRate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local elapsed = (now - lastRefill) / 1000
tokens = math.min(capacity, tokens + elapsed * refillRate)
local allowed = 0
if tokens >= 1 then
tokens = tokens - 1
allowed = 1
end
redis.call('HMSET', KEYS[1], 'tokens', tokens, 'lastRefill', now)
redis.call('PEXPIRE', KEYS[1], math.ceil(capacity / refillRate) * 1000)
return {allowed, math.floor(tokens)}
`;
const [allowed, remaining] = (await redis.eval(
script, 1, bucketKey, capacity, refillRate, now,
)) as [number, number];
return { allowed: allowed === 1, remaining };
}A bucket with capacity 100 and refill rate 10/second allows a burst of 100 requests, then sustains 10/second. This matches real usage patterns better than fixed windows.
Choosing a Strategy
| Algorithm | Burst handling | Memory | Accuracy | Complexity |
|---|---|---|---|---|
| Fixed window | Poor (boundary) | Low | Low | Simple |
| Sliding window log | Good | High | High | Medium |
| Sliding window counter | Good | Low | Medium | Medium |
| Token bucket | Excellent | Low | High | Medium |
Token bucket is the right default for most APIs. Fixed window is acceptable for internal services where precision doesn't matter.
Response Headers and 429 Handling
Always communicate rate limit status through headers. Well-behaved clients use these to self-throttle.
function rateLimitResponse(
res: Response,
limit: number,
remaining: number,
resetAt: number,
) {
res.setHeader("X-RateLimit-Limit", limit);
res.setHeader("X-RateLimit-Remaining", Math.max(0, remaining));
res.setHeader("X-RateLimit-Reset", Math.ceil(resetAt / 1000));
if (remaining < 0) {
res.setHeader("Retry-After", Math.ceil((resetAt - Date.now()) / 1000));
return res.status(429).json({
error: {
code: "RATE_LIMITED",
message: "Too many requests. Please retry after the Retry-After period.",
},
});
}
}Tiered Rate Limits
Different endpoints have different cost profiles. A search endpoint hitting a full-text index is more expensive than a status check.
const rateLimits = {
"GET /api/status": { capacity: 1000, refillRate: 100 },
"GET /api/search": { capacity: 20, refillRate: 5 },
"POST /api/orders": { capacity: 10, refillRate: 2 },
"POST /api/auth/login": { capacity: 5, refillRate: 1 },
} as const;
function getRateLimit(method: string, path: string) {
const key = `${method} ${path}`;
return rateLimits[key] ?? { capacity: 100, refillRate: 20 };
}Authentication endpoints deserve the tightest limits — they're the primary target for brute force attacks.
Key Takeaways
- Token bucket is the best default — it handles bursts naturally while enforcing average rates
- Fixed windows have a boundary problem — clients can double their effective rate at window edges
- Use Lua scripts in Redis for atomic multi-step rate limiting operations
- Always return rate limit headers —
X-RateLimit-RemainingandRetry-Afterhelp clients self-throttle - Tier rate limits by endpoint cost — expensive operations get tighter limits
- Authentication endpoints need the tightest limits — they're the primary brute force target
