Distributed Locks: Coordinating Work Across Multiple Nodes

Horizontal scaling solves throughput problems but introduces a new class of bugs: two instances of the same service doing the same work simultaneously. Sending a duplicate email, double-charging a customer, or corrupting shared state — these are race conditions at the infrastructure level, and a single database SELECT ... FOR UPDATE won't save you when the competing process lives on a different machine.

Distributed locks are the tool for this. They're also one of the most misused primitives in backend engineering. The failure modes are subtle, the literature is full of controversy (see the Redlock debate), and most tutorials stop before covering the parts that actually matter in production — lock expiration, renewal, and what happens when the lock holder dies.