The DataLoader Pattern: Killing N+1 Queries Without an ORM
N+1 is not an ORM problem — implement the DataLoader batching pattern in TypeScript to collapse redundant fetches across any source, microservices included.
The N+1 query problem gets blamed on ORMs. Fix your lazy loading, add eager: true, reach for a JOIN. But N+1 isn't an ORM problem — it's a structural problem that surfaces whenever data is fetched inside a loop. GraphQL resolvers, REST endpoints composing multiple services, React Server Components making per-row database calls: the pattern is everywhere, and the ORM is just the messenger.
The fix isn't always a JOIN. Sometimes data lives in different services. Sometimes you're calling a third-party API. The DataLoader pattern — originally from Facebook's GraphQL infrastructure — solves this at the right abstraction layer, without collapsing your resolver boundaries.
What N+1 Actually Looks Like
Here's a realistic example: a GraphQL resolver returning a list of blog posts, each with an author attached.
// ❌ N+1 — authorResolver fires once per post, 20 posts = 21 queries
const postsResolver = async () => {
return db.query<Post[]>("SELECT * FROM posts ORDER BY created_at DESC LIMIT 20");
};
const authorResolver = async (post: Post): Promise<User> => {
return db.queryOne<User>("SELECT * FROM users WHERE id = $1", [post.authorId]);
};Each resolver looks completely reasonable in isolation. The problem only emerges at runtime, under load. Twenty posts yields twenty author queries on top of the initial list query.
The naive fix works when data lives in the same database:
// ✅ For same-DB data: batch in the parent, build a lookup map
const postsWithAuthorsResolver = async () => {
const posts = await db.query<Post[]>(
"SELECT * FROM posts ORDER BY created_at DESC LIMIT 20"
);
const authorIds = [...new Set(posts.map((p) => p.authorId))];
const authors = await db.query<User[]>(
"SELECT * FROM users WHERE id = ANY($1)",
[authorIds]
);
const byId = new Map(authors.map((a) => [a.id, a]));
return posts.map((p) => ({ ...p, author: byId.get(p.authorId) }));
};But this breaks the resolver separation that makes GraphQL composable. And it falls apart entirely when the author data comes from a separate user service.
The Batching Primitive
The core insight is to defer individual fetches until the end of the current event loop tick, then flush them all as a single batch. This is exactly what queueMicrotask was designed for.
type BatchFn<K, V> = (keys: readonly K[]) => Promise<Map<K, V>>;
class DataLoader<K, V> {
private readonly batchFn: BatchFn<K, V>;
private queue: Array<{
key: K;
resolve: (value: V) => void;
reject: (error: Error) => void;
}> = [];
private scheduled = false;
private cache = new Map<K, Promise<V>>();
constructor(batchFn: BatchFn<K, V>) {
this.batchFn = batchFn;
}
load(key: K): Promise<V> {
// Return cached promise — same key in one request never fires twice
const cached = this.cache.get(key);
if (cached) return cached;
const promise = new Promise<V>((resolve, reject) => {
this.queue.push({ key, resolve, reject });
});
this.cache.set(key, promise);
if (!this.scheduled) {
this.scheduled = true;
queueMicrotask(() => this.flush());
}
return promise;
}
private async flush(): Promise<void> {
const batch = this.queue.splice(0);
this.scheduled = false;
const keys = batch.map((item) => item.key);
try {
const results = await this.batchFn(keys);
for (const { key, resolve, reject } of batch) {
const value = results.get(key);
if (value !== undefined) {
resolve(value);
} else {
reject(new Error(`DataLoader: no result for key "${String(key)}"`));
}
}
} catch (err) {
for (const { reject } of batch) {
reject(err instanceof Error ? err : new Error(String(err)));
}
}
}
}load(key) is called N times across different resolvers. All calls within the same microtask checkpoint are queued. flush() fires exactly once, sends one batched request, then fans the results back out to the individual waiting promises.
Wiring It to a Real Data Source
The batch function is where the strategy lives. Here's one backed by PostgreSQL:
const batchUsers: BatchFn<string, User> = async (userIds) => {
const rows = await db.query<User[]>(
"SELECT * FROM users WHERE id = ANY($1)",
[userIds]
);
return new Map(rows.map((u) => [u.id, u]));
};
const userLoader = new DataLoader(batchUsers);
// Each resolver calls load() — batching is transparent
const authorResolver = async (post: Post): Promise<User> => {
return userLoader.load(post.authorId);
};For a microservice call, the batch function looks identical — just swap the SQL query for an HTTP request:
const batchUsersFromService: BatchFn<string, User> = async (userIds) => {
const response = await fetch(`${USER_SERVICE_URL}/users/batch`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ ids: [...userIds] }),
});
if (!response.ok) {
throw new Error(`User service responded with ${response.status}`);
}
const users: User[] = await response.json();
return new Map(users.map((u) => [u.id, u]));
};The resolver code doesn't change. The batching strategy is fully encapsulated in the loader's batch function.
Per-Request Scoping
There's a subtle danger in the implementation above: the cache map lives on the loader instance. Create a loader at module scope, and cached data from one request leaks into the next.
Never share a DataLoader instance across requests. The per-request cache is a feature — but only if the loader is recreated for every request lifecycle.
The standard pattern is AsyncLocalStorage to scope loaders to a request:
import { AsyncLocalStorage } from "node:async_hooks";
interface RequestLoaders {
user: DataLoader<string, User>;
post: DataLoader<string, Post>;
}
const loaderStorage = new AsyncLocalStorage<RequestLoaders>();
// Express / Fastify middleware — fresh loaders per request
export function attachLoaders(
req: Request,
res: Response,
next: NextFunction
): void {
loaderStorage.run(
{
user: new DataLoader(batchUsers),
post: new DataLoader(batchPosts),
},
next
);
}
// Anywhere downstream in the call stack
export function getLoaders(): RequestLoaders {
const store = loaderStorage.getStore();
if (!store) throw new Error("getLoaders() called outside request context");
return store;
}Fresh loaders per request, no cross-contamination, automatic cache deduplication within a single request lifetime.
When Batching Isn't the Right Tool
DataLoader collapses N+1 into 1, but sometimes even a single batched query is more than you need — or less than you need. Knowing when to reach for it matters.
| Pattern | Best for |
|---|---|
| DataLoader (per-request cache) | Resolving relationships in GraphQL/REST responses |
| Redis with TTL | Cross-request data that changes infrequently |
SQL JOIN | Co-located data with predictable, uniform access patterns |
| Materialized view | Read-heavy aggregations refreshed on a schedule |
| In-process LRU cache | Hot reference data (feature flags, config, enums) |
Don't reach for DataLoader when a JOIN is simpler — it earns its complexity when data spans service boundaries or when resolver isolation matters more than raw query efficiency. The pattern also assumes your batch function can accept arbitrary key sets; if the downstream API only supports fetching single records, you're solving the wrong problem.
Key Takeaways
- N+1 is structural, not ORM-specific — it emerges any time data is fetched inside a loop, regardless of the data layer
- Defer, then flush —
queueMicrotaskcollects allload()calls within one tick into a single batch; no manual coordination required - The batch function is the adapter — the same
DataLoaderimplementation works with SQL, HTTP, Redis, or any async source - Scope loaders to the request — a module-level loader will serve stale cached data to subsequent requests; use
AsyncLocalStorageto wire fresh instances - Cache deduplication comes free — calling
loader.load(id)twice with the same key returns the same promise, preventing redundant in-flight requests
