Distributed Caching: Consistency, Invalidation, Failure
Distributed caching patterns in depth: cache-aside, write-through, read-through and the invalidation strategies that survive real consistency demands.
Caching is the most powerful performance optimization in distributed systems and the most dangerous source of bugs. A well-implemented cache reduces database load by 90%. A poorly implemented one serves stale data for hours while developers wonder why users see yesterday's prices.
The difference between these outcomes isn't the cache technology—it's the caching pattern. Each pattern makes different trade-offs between consistency, performance, and complexity.
Cache-Aside: The Default Pattern
Cache-aside (lazy loading) is the most common pattern. The application manages the cache explicitly: check the cache, hit the database on miss, populate the cache.
// ❌ Naive cache-aside with race condition
async function getUser(id: string): Promise<User> {
const cached = await redis.get(`user:${id}`);
if (cached) return JSON.parse(cached);
const user = await db.users.findById(id);
await redis.set(`user:${id}`, JSON.stringify(user));
// Race: another request might have updated the DB between
// our read and our cache write, caching stale data
return user;
}// ✅ Cache-aside with TTL and stampede protection
import { Redis } from "ioredis";
class CacheAside<T> {
constructor(
private redis: Redis,
private prefix: string,
private ttlSeconds: number = 300
) {}
async get(
key: string,
fetcher: () => Promise<T>
): Promise<T> {
const cacheKey = `${this.prefix}:${key}`;
// Try cache first
const cached = await this.redis.get(cacheKey);
if (cached) {
return JSON.parse(cached) as T;
}
// Stampede protection: only one request populates cache
const lockKey = `${cacheKey}:lock`;
const acquired = await this.redis.set(
lockKey, "1", "EX", 10, "NX"
);
if (!acquired) {
// Another request is fetching; wait and retry cache
await new Promise(r => setTimeout(r, 100));
const retried = await this.redis.get(cacheKey);
if (retried) return JSON.parse(retried) as T;
}
// Fetch from source
const value = await fetcher();
// Cache with TTL
await this.redis.set(
cacheKey,
JSON.stringify(value),
"EX",
this.ttlSeconds
);
// Release lock
await this.redis.del(lockKey);
return value;
}
async invalidate(key: string): Promise<void> {
await this.redis.del(`${this.prefix}:${key}`);
}
}
// Usage
const userCache = new CacheAside<User>(redis, "user", 300);
const user = await userCache.get("user-123", () =>
db.users.findById("user-123")
);The lock prevents cache stampede: when thousands of requests hit an expired cache key simultaneously, only one request fetches from the database. The rest wait briefly and get the freshly cached result.
Write-Through: Consistency at the Cost of Latency
Write-through caching updates the cache synchronously with every database write. Reads are always fast, but writes pay the cost of both storage operations.
// ❌ Cache and DB can get out of sync
async function updateUser(id: string, data: Partial<User>) {
await db.users.update(id, data); // DB updated
await redis.del(`user:${id}`); // Cache invalidated
// If the app crashes between these lines, cache is stale
}// ✅ Write-through with atomic-like guarantees
class WriteThroughCache<T> {
constructor(
private redis: Redis,
private prefix: string,
private ttlSeconds: number
) {}
async write(
key: string,
value: T,
dbWriter: (value: T) => Promise<void>
): Promise<void> {
const cacheKey = `${this.prefix}:${key}`;
// Write to DB first (source of truth)
await dbWriter(value);
// Then update cache
await this.redis.set(
cacheKey,
JSON.stringify(value),
"EX",
this.ttlSeconds
);
}
async read(key: string): Promise<T | null> {
const cacheKey = `${this.prefix}:${key}`;
const cached = await this.redis.get(cacheKey);
return cached ? (JSON.parse(cached) as T) : null;
}
}
// Usage
const productCache = new WriteThroughCache<Product>(redis, "product", 600);
await productCache.write(
"prod-456",
updatedProduct,
async (product) => {
await db.products.update(product.id, product);
}
);Write-through guarantees that the cache always has fresh data after a write. The trade-off is higher write latency (two operations instead of one) and the risk of caching data for keys that are rarely read.
Read-Through With Stale-While-Revalidate
Stale-while-revalidate serves slightly stale data immediately while refreshing the cache in the background. This eliminates cache-miss latency for users while keeping data reasonably fresh.
interface CacheEntry<T> {
value: T;
cachedAt: number;
staleAfter: number;
expireAfter: number;
}
class StaleWhileRevalidateCache<T> {
private refreshing = new Set<string>();
constructor(
private redis: Redis,
private prefix: string,
private freshSeconds: number = 60,
private staleSeconds: number = 300
) {}
async get(
key: string,
fetcher: () => Promise<T>
): Promise<T> {
const cacheKey = `${this.prefix}:${key}`;
const raw = await this.redis.get(cacheKey);
if (raw) {
const entry: CacheEntry<T> = JSON.parse(raw);
const now = Date.now();
if (now < entry.staleAfter) {
// Fresh: return immediately
return entry.value;
}
if (now < entry.expireAfter) {
// Stale but usable: return immediately, refresh in background
this.refreshInBackground(key, cacheKey, fetcher);
return entry.value;
}
}
// Expired or missing: fetch synchronously
return this.fetchAndCache(key, cacheKey, fetcher);
}
private async fetchAndCache(
key: string,
cacheKey: string,
fetcher: () => Promise<T>
): Promise<T> {
const value = await fetcher();
const now = Date.now();
const entry: CacheEntry<T> = {
value,
cachedAt: now,
staleAfter: now + this.freshSeconds * 1000,
expireAfter: now + this.staleSeconds * 1000,
};
await this.redis.set(
cacheKey,
JSON.stringify(entry),
"EX",
this.staleSeconds
);
return value;
}
private refreshInBackground(
key: string,
cacheKey: string,
fetcher: () => Promise<T>
): void {
if (this.refreshing.has(key)) return; // Already refreshing
this.refreshing.add(key);
this.fetchAndCache(key, cacheKey, fetcher)
.finally(() => this.refreshing.delete(key));
}
}This pattern is excellent for data that changes periodically but doesn't need real-time accuracy—product catalogs, user profiles, configuration settings. Users always get a fast response, and the data is never more than staleSeconds old.
Multi-Level Caching
Production applications often use multiple cache layers: in-process memory, distributed cache (Redis), and CDN. Each layer serves different access patterns.
class MultiLevelCache<T> {
private l1: Map<string, { value: T; expires: number }> = new Map();
private l1MaxSize: number;
constructor(
private redis: Redis,
private prefix: string,
l1MaxSize: number = 1000,
private l1TtlMs: number = 10000,
private l2TtlSeconds: number = 300
) {
this.l1MaxSize = l1MaxSize;
}
async get(
key: string,
fetcher: () => Promise<T>
): Promise<T> {
// L1: In-process memory (microseconds)
const l1Entry = this.l1.get(key);
if (l1Entry && l1Entry.expires > Date.now()) {
return l1Entry.value;
}
// L2: Redis (milliseconds)
const cacheKey = `${this.prefix}:${key}`;
const l2Value = await this.redis.get(cacheKey);
if (l2Value) {
const parsed = JSON.parse(l2Value) as T;
this.setL1(key, parsed);
return parsed;
}
// L3: Database (tens of milliseconds)
const value = await fetcher();
this.setL1(key, value);
await this.redis.set(
cacheKey,
JSON.stringify(value),
"EX",
this.l2TtlSeconds
);
return value;
}
private setL1(key: string, value: T): void {
// Simple eviction: remove oldest when full
if (this.l1.size >= this.l1MaxSize) {
const oldest = this.l1.keys().next().value;
if (oldest !== undefined) {
this.l1.delete(oldest);
}
}
this.l1.set(key, {
value,
expires: Date.now() + this.l1TtlMs,
});
}
async invalidate(key: string): Promise<void> {
this.l1.delete(key);
await this.redis.del(`${this.prefix}:${key}`);
// Note: other application instances still have L1 cache
// Publish invalidation event for cluster-wide L1 flush
await this.redis.publish(
`${this.prefix}:invalidate`,
key
);
}
}The L1 cache (in-process) has a very short TTL because it can't be invalidated across instances. The L2 cache (Redis) has a longer TTL because it's shared. This layered approach handles the hot-path reads in microseconds while maintaining reasonable consistency.
Handling Cache Failures Gracefully
Cache failures shouldn't cascade into application failures. When Redis goes down, the application should degrade to database-direct reads, not crash.
class ResilientCache<T> {
private circuitOpen = false;
private failureCount = 0;
private lastFailure = 0;
constructor(
private redis: Redis,
private prefix: string,
private ttlSeconds: number,
private failureThreshold: number = 5,
private resetTimeMs: number = 30000
) {}
async get(
key: string,
fetcher: () => Promise<T>
): Promise<T> {
// Check circuit breaker
if (this.circuitOpen) {
if (Date.now() - this.lastFailure > this.resetTimeMs) {
this.circuitOpen = false;
this.failureCount = 0;
} else {
// Circuit open: skip cache entirely
return fetcher();
}
}
try {
const cacheKey = `${this.prefix}:${key}`;
const cached = await this.redis.get(cacheKey);
if (cached) {
this.failureCount = 0;
return JSON.parse(cached) as T;
}
const value = await fetcher();
// Best-effort cache write
this.redis
.set(cacheKey, JSON.stringify(value), "EX", this.ttlSeconds)
.catch(() => this.recordFailure());
return value;
} catch {
this.recordFailure();
return fetcher();
}
}
private recordFailure(): void {
this.failureCount++;
this.lastFailure = Date.now();
if (this.failureCount >= this.failureThreshold) {
this.circuitOpen = true;
}
}
}The circuit breaker prevents repeated cache failures from adding latency to every request. After five Redis failures, the circuit opens and the application goes directly to the database until Redis recovers.
Key Takeaways
Choosing a caching pattern isn't about picking the "best" one—it's about matching the pattern to your consistency requirements. Cache-aside works for most read-heavy workloads. Write-through works when reads must always see the latest write. Stale-while-revalidate works when near-real-time is good enough and latency is critical.
The patterns that cause the most production incidents are those that ignore failure modes. Always implement stampede protection for cache-aside, always handle cache failures with circuit breakers, and always assume that multi-instance deployments need cross-instance invalidation. The cache that performs perfectly but serves stale data after an update is worse than no cache at all.
