Designing Idempotent APIs: Safely Retrying Failed Requests
Build idempotent API endpoints that retry safely using idempotency keys, database constraints and state machines — payments, orders and webhooks.
Network requests fail. Clients time out, connections drop, load balancers reset, and servers restart mid-request. When a client doesn't receive a response, it has no way to know whether the request was processed—the server might have completed the operation but the response was lost. Without idempotency, retrying that request creates a duplicate: a double charge, a duplicate order, or an extra message sent.
Idempotent APIs guarantee that executing the same request multiple times produces the same result as executing it once. GET and DELETE are naturally idempotent. POST and PATCH are not—they need explicit design to become safe for retries.
The Idempotency Key Pattern
The client sends a unique key with each request. The server stores the result keyed by this identifier and returns the cached result on retries.
// ❌ Non-idempotent payment endpoint
app.post("/api/payments", async (req, res) => {
const { userId, amount, currency } = req.body;
// If client retries after timeout: double charge
const charge = await stripe.charges.create({
amount,
currency,
customer: userId,
});
await db.createPayment({
userId,
amount,
stripeChargeId: charge.id,
});
res.status(201).json({ paymentId: charge.id });
});// ✅ Idempotent payment endpoint
app.post("/api/payments", async (req, res) => {
const idempotencyKey = req.headers[
"idempotency-key"
] as string;
if (!idempotencyKey) {
res.status(400).json({
error: "Idempotency-Key header is required",
});
return;
}
// Check for existing result
const existing = await db.getIdempotencyRecord(
idempotencyKey
);
if (existing) {
if (existing.status === "processing") {
// Request is still being processed
res.status(409).json({
error: "Request is still being processed",
retryAfter: 2,
});
return;
}
// Return cached response
res.status(existing.statusCode).json(
existing.responseBody
);
return;
}
// Lock the idempotency key to prevent races
const locked = await db.createIdempotencyRecord({
key: idempotencyKey,
status: "processing",
requestBody: req.body,
createdAt: new Date(),
expiresAt: new Date(Date.now() + 86_400_000),
});
if (!locked) {
// Another request with same key is in flight
res.status(409).json({
error: "Duplicate request in progress",
});
return;
}
try {
const { userId, amount, currency } = req.body;
const charge = await stripe.charges.create({
amount,
currency,
customer: userId,
idempotencyKey, // Stripe supports this natively
});
const payment = await db.createPayment({
userId,
amount,
stripeChargeId: charge.id,
});
const response = { paymentId: payment.id };
// Cache the successful response
await db.updateIdempotencyRecord(idempotencyKey, {
status: "completed",
statusCode: 201,
responseBody: response,
});
res.status(201).json(response);
} catch (error) {
// Cache the error response
await db.updateIdempotencyRecord(idempotencyKey, {
status: "failed",
statusCode: 500,
responseBody: {
error: "Payment processing failed",
},
});
res.status(500).json({
error: "Payment processing failed",
});
}
});Database-Level Idempotency
For simpler cases, database constraints can enforce idempotency without an explicit key store.
// ❌ Race condition: two requests create duplicate orders
app.post("/api/orders", async (req, res) => {
const order = await db.query(
"INSERT INTO orders (user_id, product_id, quantity) VALUES ($1, $2, $3) RETURNING *",
[req.body.userId, req.body.productId, req.body.quantity]
);
res.status(201).json(order.rows[0]);
});// ✅ Unique constraint prevents duplicates
// Migration: CREATE UNIQUE INDEX
// idx_orders_idempotency
// ON orders (idempotency_key)
app.post("/api/orders", async (req, res) => {
const { userId, productId, quantity } = req.body;
const idempotencyKey = req.headers[
"idempotency-key"
] as string;
try {
const result = await db.query(
`INSERT INTO orders
(idempotency_key, user_id, product_id, quantity)
VALUES ($1, $2, $3, $4)
ON CONFLICT (idempotency_key)
DO UPDATE SET idempotency_key = EXCLUDED.idempotency_key
RETURNING *`,
[idempotencyKey, userId, productId, quantity]
);
// Whether this was an insert or no-op,
// result is the same order
res.status(201).json(result.rows[0]);
} catch (error) {
res.status(500).json({
error: "Failed to create order",
});
}
});Idempotency Middleware
Extract the idempotency logic into reusable middleware that works across endpoints.
interface IdempotencyRecord {
key: string;
status: "processing" | "completed" | "failed";
statusCode: number;
responseBody: unknown;
requestFingerprint: string;
createdAt: Date;
expiresAt: Date;
}
function idempotent(ttlMs: number = 86_400_000) {
return async (
req: Request,
res: Response,
next: NextFunction
) => {
const key = req.headers["idempotency-key"] as string;
if (!key) {
next(); // Non-idempotent by default
return;
}
// Fingerprint request to detect mismatched retries
const fingerprint = createFingerprint(
req.method,
req.path,
req.body
);
const existing = await cache.get<IdempotencyRecord>(
`idempotency:${key}`
);
if (existing) {
// Verify the request matches the original
if (existing.requestFingerprint !== fingerprint) {
res.status(422).json({
error:
"Idempotency key reused with " +
"different request parameters",
});
return;
}
if (existing.status === "processing") {
res.status(409).json({
error: "Request still processing",
retryAfter: 2,
});
return;
}
res
.status(existing.statusCode)
.json(existing.responseBody);
return;
}
// Reserve the key
await cache.set(
`idempotency:${key}`,
{
key,
status: "processing",
requestFingerprint: fingerprint,
createdAt: new Date(),
expiresAt: new Date(Date.now() + ttlMs),
} as IdempotencyRecord,
ttlMs
);
// Intercept the response to cache it
const originalJson = res.json.bind(res);
res.json = function (body: unknown) {
cache.set(
`idempotency:${key}`,
{
key,
status:
res.statusCode < 500
? "completed"
: "failed",
statusCode: res.statusCode,
responseBody: body,
requestFingerprint: fingerprint,
createdAt: new Date(),
expiresAt: new Date(Date.now() + ttlMs),
} as IdempotencyRecord,
ttlMs
);
return originalJson(body);
};
next();
};
}
function createFingerprint(
method: string,
path: string,
body: unknown
): string {
const crypto = require("node:crypto");
return crypto
.createHash("sha256")
.update(
JSON.stringify({ method, path, body })
)
.digest("hex");
}
// Usage
app.post(
"/api/payments",
idempotent(24 * 60 * 60 * 1000),
paymentHandler
);
app.post(
"/api/orders",
idempotent(1 * 60 * 60 * 1000),
orderHandler
);State Machine-Based Idempotency
For complex operations with multiple steps, a state machine ensures each step executes exactly once even across retries.
type OrderState =
| "created"
| "payment_pending"
| "payment_confirmed"
| "fulfillment_pending"
| "shipped"
| "failed";
const validTransitions: Record<
OrderState,
OrderState[]
> = {
created: ["payment_pending", "failed"],
payment_pending: ["payment_confirmed", "failed"],
payment_confirmed: ["fulfillment_pending", "failed"],
fulfillment_pending: ["shipped", "failed"],
shipped: [],
failed: [],
};
async function transitionOrder(
orderId: string,
targetState: OrderState,
action: () => Promise<void>
): Promise<boolean> {
// Atomic state transition with optimistic locking
const order = await db.query(
`SELECT id, state, version FROM orders
WHERE id = $1 FOR UPDATE`,
[orderId]
);
const currentState = order.rows[0]
.state as OrderState;
const version = order.rows[0].version;
// Already in target state? Idempotent success
if (currentState === targetState) {
return true;
}
// Validate transition
if (
!validTransitions[currentState]?.includes(
targetState
)
) {
throw new Error(
`Invalid transition: ${currentState} → ${targetState}`
);
}
// Execute the action
await action();
// Update state with version check
const result = await db.query(
`UPDATE orders
SET state = $1, version = version + 1
WHERE id = $2 AND version = $3`,
[targetState, orderId, version]
);
if (result.rowCount === 0) {
throw new Error(
"Concurrent modification detected"
);
}
return true;
}
// Usage: retry-safe order processing
async function processOrder(orderId: string) {
await transitionOrder(
orderId,
"payment_pending",
async () => {
// Reserve inventory
}
);
await transitionOrder(
orderId,
"payment_confirmed",
async () => {
// Charge payment
}
);
await transitionOrder(
orderId,
"fulfillment_pending",
async () => {
// Queue for shipping
}
);
}Key Takeaways
Idempotency keys sent by the client allow the server to detect retries and return cached responses instead of re-executing side effects—this is essential for payment processing, order creation, and any operation where duplicates cause real-world harm. Database-level constraints using ON CONFLICT clauses provide simpler idempotency for operations that can be expressed as upserts, avoiding the need for a separate idempotency key store when the natural key serves as the deduplication mechanism. Reusable middleware that intercepts responses and caches them by idempotency key should verify request fingerprints to reject reuse of the same key with different parameters, and return 409 for requests still in-flight to prevent parallel duplicate processing. State machines with optimistic locking ensure multi-step operations are idempotent by making each state transition atomic and checking whether the target state has already been reached—retrying a transition that already completed is a no-op success.
