Understanding Consensus Algorithms in Distributed Systems
A practical guide to consensus algorithms: Raft, Paxos and leader election, with real examples of when and why distributed systems need them.
When multiple servers need to agree on a single value — which node is the leader, what the current state of a replicated log is, whether a transaction should commit — they need a consensus algorithm. Without consensus, distributed systems split into partitions that disagree about reality, and your users see stale data, duplicate operations, or outright failures.
Consensus is the foundation underneath every replicated database, every distributed queue, and every coordination service. Understanding it changes how you reason about the systems you build on top of them.
Why Consensus Is Hard
In a single-server system, there is one source of truth. In a distributed system, that guarantee disappears. Servers crash, networks partition, messages arrive out of order or not at all. The FLP impossibility result proves that no deterministic algorithm can guarantee consensus in an asynchronous system where even one process might crash.
Practical consensus algorithms work around this by using timeouts and randomization — they sacrifice theoretical guarantees for real-world reliability.
// The fundamental problem consensus solves
interface ConsensusScenario {
nodes: string[];
proposedValues: Map<string, string>;
outcome: 'agree' | 'disagree' | 'stuck';
}
// ❌ Without consensus — each node believes something different
const withoutConsensus: ConsensusScenario = {
nodes: ['A', 'B', 'C'],
proposedValues: new Map([
['A', 'commit'], // A thinks the transaction committed
['B', 'abort'], // B thinks it aborted
['C', 'commit'], // C agrees with A but B disagrees
]),
outcome: 'disagree', // Split brain — data corruption
};
// ✅ With consensus — all nodes converge on one value
const withConsensus: ConsensusScenario = {
nodes: ['A', 'B', 'C'],
proposedValues: new Map([
['A', 'commit'],
['B', 'commit'], // B accepted the majority decision
['C', 'commit'],
]),
outcome: 'agree', // All nodes agree — consistency preserved
};Raft: Consensus Made Understandable
Raft was designed specifically to be understandable. It decomposes consensus into three sub-problems: leader election, log replication, and safety. Each part can be reasoned about independently.
// Simplified Raft node state
type NodeRole = 'follower' | 'candidate' | 'leader';
interface RaftNode {
id: string;
role: NodeRole;
currentTerm: number;
votedFor: string | null;
log: LogEntry[];
commitIndex: number;
lastApplied: number;
}
interface LogEntry {
term: number;
index: number;
command: string;
}
// Leader election state machine
function handleElectionTimeout(node: RaftNode): RaftNode {
// Follower hasn't heard from leader — become candidate
return {
...node,
role: 'candidate',
currentTerm: node.currentTerm + 1,
votedFor: node.id, // Vote for self
};
}
function handleVoteRequest(
node: RaftNode,
candidateId: string,
candidateTerm: number,
candidateLogLength: number
): { granted: boolean; updatedNode: RaftNode } {
// Reject if candidate's term is stale
if (candidateTerm < node.currentTerm) {
return { granted: false, updatedNode: node };
}
// Grant vote if we haven't voted yet and candidate's log
// is at least as up-to-date as ours
const canVote = node.votedFor === null || node.votedFor === candidateId;
const logUpToDate = candidateLogLength >= node.log.length;
if (canVote && logUpToDate) {
return {
granted: true,
updatedNode: {
...node,
currentTerm: candidateTerm,
votedFor: candidateId,
role: 'follower',
},
};
}
return { granted: false, updatedNode: node };
}The election process works in rounds called terms. When a follower does not receive a heartbeat from the leader within its election timeout, it increments its term and requests votes. A candidate that receives votes from a majority becomes the leader. Because only one leader can win per term, the cluster always has at most one leader.
Log Replication in Practice
Once a leader is elected, it accepts client requests and replicates them to followers. A log entry is committed only after a majority of nodes have written it to their logs.
// Leader replicating entries to followers
interface AppendEntriesRequest {
term: number;
leaderId: string;
prevLogIndex: number;
prevLogTerm: number;
entries: LogEntry[];
leaderCommit: number;
}
interface AppendEntriesResponse {
term: number;
success: boolean;
matchIndex: number;
}
function leaderReplicateEntry(
leader: RaftNode,
command: string,
followers: string[]
): void {
const entry: LogEntry = {
term: leader.currentTerm,
index: leader.log.length,
command,
};
// Append to leader's own log immediately
leader.log.push(entry);
// Send to all followers in parallel
const ackCount = 1; // Leader counts as one acknowledgment
const majority = Math.floor((followers.length + 1) / 2) + 1;
// When majority acknowledges:
// leader.commitIndex = entry.index;
// Apply command to state machine
// Respond to client with success
console.log(
`Entry ${entry.index} needs ${majority} acks ` +
`(have ${ackCount}, waiting for ${majority - ackCount} more)`
);
}// ❌ Committing before majority acknowledgment
function unsafeCommit(leader: RaftNode, entry: LogEntry): void {
leader.commitIndex = entry.index; // Committed with only leader's copy
applyToStateMachine(entry); // Could lose data if leader crashes
}
// ✅ Committing only after majority acknowledgment
function safeCommit(
leader: RaftNode,
entry: LogEntry,
ackCount: number,
clusterSize: number
): boolean {
const majority = Math.floor(clusterSize / 2) + 1;
if (ackCount >= majority) {
leader.commitIndex = entry.index;
applyToStateMachine(entry);
return true; // Safe to respond to client
}
return false; // Not yet committed — keep waiting
}
function applyToStateMachine(entry: LogEntry): void {
console.log(`Applying: ${entry.command}`);
}The majority requirement is what makes Raft fault-tolerant. In a 5-node cluster, 2 nodes can fail and the remaining 3 still form a majority. The committed entries are guaranteed to survive because at least one node in any future majority will have them.
Leader Election Timing
Election timeouts must be randomized to prevent split votes — where two candidates start elections simultaneously and neither gets a majority.
// Election timeout configuration
interface ElectionConfig {
minTimeoutMs: number;
maxTimeoutMs: number;
heartbeatIntervalMs: number;
}
// Rule: heartbeat << election timeout
// This ensures followers hear from the leader
// well before they start unnecessary elections
const config: ElectionConfig = {
minTimeoutMs: 150, // Minimum election timeout
maxTimeoutMs: 300, // Maximum election timeout
heartbeatIntervalMs: 50, // Leader sends heartbeats at this interval
};
function randomElectionTimeout(config: ElectionConfig): number {
const range = config.maxTimeoutMs - config.minTimeoutMs;
return config.minTimeoutMs + Math.floor(Math.random() * range);
}
// In production systems like etcd:
// - Heartbeat interval: 100ms
// - Election timeout: 1000-2000ms
// - The 10x ratio gives plenty of margin for network delaysThe randomization breaks symmetry. If all nodes used the same timeout, they would all become candidates simultaneously, split the vote, time out again, and repeat — a livelock. Randomized timeouts ensure that usually one node times out first and wins the election before others start.
When to Use Consensus vs. When to Avoid It
Consensus is expensive — it requires network round trips and majority quorums. Not every system needs it.
# When consensus IS needed:
leader_election:
examples: ["database primary selection", "distributed lock service"]
why: "Only one node should act as leader at a time"
replicated_state_machine:
examples: ["etcd", "ZooKeeper", "CockroachDB"]
why: "All replicas must apply the same operations in the same order"
distributed_transactions:
examples: ["two-phase commit coordinator", "saga orchestrator"]
why: "All participants must agree on commit or abort"
# When consensus is OVERKILL:
caching:
alternative: "Eventual consistency — stale cache entries are acceptable"
analytics:
alternative: "Approximate counts — exact agreement not required"
event_streaming:
alternative: "At-least-once delivery — consumers handle duplicates"
dns_propagation:
alternative: "Eventual consistency — TTLs handle staleness"If your system can tolerate temporary disagreement between nodes, you do not need consensus. Eventual consistency with conflict resolution is simpler, faster, and more available. Reserve consensus for cases where disagreement causes correctness violations — such as two nodes both believing they are the database primary.
Key Takeaways
- Consensus solves the agreement problem — multiple nodes converging on a single value despite crashes and network failures
- Raft decomposes consensus into leader election, log replication, and safety — each part is independently understandable
- Majority quorums provide fault tolerance — a 5-node cluster survives 2 failures while maintaining consistency
- Randomized election timeouts prevent split votes — symmetry breaking is essential for liveness
- Consensus is expensive — reserve it for cases where disagreement causes correctness problems, not performance
- Production systems (etcd, ZooKeeper, CockroachDB) handle the complexity — understand the principles so you configure them correctly
