System Design Fundamentals Every Developer Should Know
The core concepts behind scalable system design — load balancing, caching, database scaling, queues, and the tradeoffs that guide architectural decisions.
Why Every Developer Needs System Design Knowledge
You don't need to be an architect to benefit from system design thinking. Understanding how the pieces fit together makes you a better developer at every level — it changes how you write code, how you debug production issues, and how you evaluate tradeoffs in daily decisions.
Concept 1: Horizontal vs Vertical Scaling
Vertical scaling — make the machine bigger (more CPU, RAM). Simple, but has a ceiling and a single point of failure.
Horizontal scaling — add more machines. Requires your application to be stateless.
Vertical: Horizontal:
[Big Server] [Server] [Server] [Server]
| | | |
[DB] [Load Balancer]
|
[DB]
The key insight: stateless applications (where no user data is stored in application memory) can scale horizontally. If your server stores session data in memory, you can't route the same user to different servers.
Solution: externalize state.
// ❌ In-memory session — breaks horizontal scaling
const sessions = new Map<string, Session>();
app.use((req, res, next) => {
req.session = sessions.get(req.cookies.sessionId);
next();
});
// ✅ Externalized session — any server can serve any user
import { createClient } from "redis";
const redis = createClient({ url: process.env.REDIS_URL });
app.use(
session({
store: new RedisStore({ client: redis }),
secret: process.env.SESSION_SECRET,
}),
);Concept 2: Caching Layers
Caching is the most impactful performance optimization available at the system level. There are four places to cache:
Browser Cache → CDN → Application Cache (Redis) → Database Query Cache
↑ ↑ ↑ ↑
Static assets Static + API Computed results Query results
ms access ~10ms ~1ms ~5ms
The hardest part isn't implementing a cache — it's cache invalidation.
// Cache-aside pattern — the most common approach
async function getProduct(id: string): Promise<Product> {
const cacheKey = `product:${id}`;
// 1. Try cache
const cached = await redis.get(cacheKey);
if (cached) return JSON.parse(cached);
// 2. Cache miss — fetch from database
const product = await db.product.findUnique({ where: { id } });
if (!product) throw new NotFoundError();
// 3. Populate cache with TTL
await redis.setEx(cacheKey, 3600, JSON.stringify(product)); // 1 hour TTL
return product;
}
// Invalidate on mutation
async function updateProduct(id: string, data: Partial<Product>) {
const product = await db.product.update({ where: { id }, data });
await redis.del(`product:${id}`); // Invalidate cache entry
return product;
}Cache stampede: when the cache expires, all requests miss simultaneously and hammer the database. Mitigate with probabilistic early expiration or distributed locks.
Concept 3: Message Queues
Queues decouple producers from consumers, enabling:
- Async processing — return a response immediately, process in background
- Load leveling — absorb traffic spikes without overwhelming downstream services
- Retry logic — failed jobs can be retried automatically
// Without a queue — user waits for email to send
app.post("/api/register", async (req, res) => {
const user = await createUser(req.body);
await sendWelcomeEmail(user.email); // Slow — blocks response
res.json({ user });
});
// With a queue — response is immediate, email is async
app.post("/api/register", async (req, res) => {
const user = await createUser(req.body);
await queue.add("send-welcome-email", { userId: user.id }); // Fast — non-blocking
res.json({ user });
});
// Worker processes jobs independently
queue.process("send-welcome-email", async (job) => {
const user = await db.user.findUnique({ where: { id: job.data.userId } });
await emailService.sendWelcome(user);
});Concept 4: Database Scaling
When a single database can't handle the load:
Read Replicas — replicate writes to read-only replicas. Route read-heavy queries to replicas.
// Primary for writes, replica for reads
const writeDb = new PrismaClient({ datasources: { db: { url: PRIMARY_URL } } });
const readDb = new PrismaClient({ datasources: { db: { url: REPLICA_URL } } });
async function getUserDashboard(userId: string) {
// Read from replica — ok if slightly stale
return readDb.user.findUnique({
where: { id: userId },
include: { recentOrders: { take: 10 } },
});
}
async function updateUserProfile(userId: string, data: Partial<User>) {
// Write to primary — must be consistent
return writeDb.user.update({ where: { id: userId }, data });
}Sharding — partition data across multiple databases by a shard key (e.g., user ID ranges). Adds significant operational complexity — use read replicas first.
Concept 5: The CAP Theorem in Practice
In a distributed system, you can only guarantee two of three:
- Consistency — every read sees the most recent write
- Availability — every request receives a response
- Partition tolerance — the system works despite network failures
Network failures happen. You must choose between Consistency and Availability during partitions.
Most web applications should choose Availability + Eventual Consistency:
// Eventual consistency — show cached data, sync in background
async function getInventoryCount(productId: string) {
// Return cached count — may be slightly stale
const cached = await redis.get(`inventory:${productId}`);
if (cached) return parseInt(cached);
// Fallback to database
const product = await db.product.findUnique({
where: { id: productId },
select: { inventoryCount: true },
});
await redis.setEx(
`inventory:${productId}`,
30,
String(product.inventoryCount),
);
return product.inventoryCount;
}Banking transactions and medical records need strict consistency. Product inventory counts can tolerate a 30-second lag.
The Mental Model
System design is about understanding where the bottlenecks are and which tradeoffs are acceptable for your use case.
Questions to ask for every system:
- What is the read/write ratio?
- What is the acceptable latency?
- What happens when any component fails?
- What does the data access pattern look like?
- What are the consistency requirements?
Start simple. Add complexity only when you have evidence that the simpler solution is a bottleneck. Premature optimization in system design — over-engineering for scale you don't have yet — is as dangerous as writing premature micro-optimizations in code.
