Building a Personal Knowledge Management System
How to build a sustainable knowledge management system as a developer: note-taking frameworks, linking strategies and retrieval that compounds.
Every developer reads documentation, debugs obscure errors, and discovers better patterns. Most of this knowledge evaporates within weeks. A personal knowledge management (PKM) system captures, organizes, and resurfaces what you learn so it compounds instead of fading.
This is not about choosing the right app. Tools matter less than the system you build on top of them. A plain text file structure with good habits beats a sophisticated tool with no process.
The Capture Habit
The most important part of knowledge management is capturing information at the moment you encounter it. If you wait even 30 minutes, most context is lost.
## Capture Template (daily-notes/2021-06-25.md)
### Debug: PostgreSQL connection pool exhaustion
- **Context**: Production alert at 2pm, API response times spiked to 8s
- **Root cause**: Long-running analytics query was holding connections
for 45+ seconds, exhausting the 20-connection pool
- **Fix**: Separate read replica connection pool for analytics queries
with statement_timeout of 30s
- **Ref**: https://wiki.postgresql.org/wiki/Number_Of_Database_Connections
- **Tags**: #postgresql #connection-pool #production-incident
### TIL: CSS `aspect-ratio` works on replaced elements
- Using `aspect-ratio: 16/9` on an img tag prevents layout shift
without needing the padding-bottom hack
- Still need width OR height set — aspect-ratio calculates the other
- **Tags**: #css #layout-shift #performanceThe capture template has five fields: context (why you looked this up), content (what you learned), source (where you found it), connections (what it relates to), and tags.
Organizing with Maps of Content
Flat tag systems break down at scale. Maps of Content (MOCs) are index notes that curate related notes into a navigable structure.
## Database Performance MOC (mocs/database-performance.md)
### Connection Management
- [[connection-pooling-essentials]] — pool sizing, pgBouncer config
- [[database-connection-exhaustion-debug]] — production incident, read replica fix
- [[connection-timeout-strategies]] — statement_timeout, idle_in_transaction
### Query Optimization
- [[sql-query-optimization-guide]] — EXPLAIN ANALYZE, index selection
- [[slow-query-log-analysis]] — pg_stat_statements, identifying hot paths
- [[n-plus-one-query-patterns]] — ORM pitfalls, dataloader pattern
### Indexing
- [[database-indexing-deep-dive]] — B-tree, GIN, partial indexes
- [[composite-index-ordering]] — column order matters, left-prefix rule
- [[index-bloat-management]] — REINDEX, pg_repack, monitoring
### Scaling Patterns
- [[read-replica-architecture]] — connection routing, replication lag
- [[database-sharding-strategies]] — hash, range, directory-based
- [[connection-pool-per-service]] — microservices isolation patternMOCs work because they mirror how your brain organizes knowledge — not in rigid hierarchies, but in contextual clusters. A note about connection pooling appears in the Database Performance MOC and might also appear in a Production Incidents MOC.
The Zettelkasten Link Strategy
Individual notes become powerful when linked. The Zettelkasten method creates a network where every note connects to related ideas.
## Note: Statement Timeout Strategy (notes/statement-timeout-strategy.md)
**ID**: 2021-06-25-1403
**Tags**: #postgresql #timeout #resilience
Setting `statement_timeout` at the connection pool level prevents
any single query from holding resources indefinitely.
```sql
-- Per-connection pool timeout (set in pgBouncer or application config)
ALTER ROLE analytics_reader SET statement_timeout = '30s';
ALTER ROLE api_reader SET statement_timeout = '5s';
-- Per-query override when needed
SET LOCAL statement_timeout = '60s';
SELECT * FROM expensive_analytics_view;Different roles get different timeouts based on their expected query patterns. The API pool gets a strict 5s timeout because any query taking longer than that should be moved to a background job.
Links:
- Relates to: [[connection-pooling-essentials]] — pool configuration
- Triggered by: [[database-connection-exhaustion-debug]] — discovery context
- Supports: [[circuit-breaker-pattern]] — timeout is a form of circuit breaking
- See also: [[timeout-retry-backoff-pattern]] — what happens after timeout
Each note links forward (what this enables) and backward (what led to this). Over time, heavily-linked notes reveal the core concepts in your knowledge base.
## Retrieval Workflows
Knowledge you cannot find is knowledge you do not have. Build retrieval into your daily workflow.
```typescript
// Simple local search script for markdown knowledge base
import { readdir, readFile } from 'fs/promises';
import { join } from 'path';
interface SearchResult {
file: string;
line: number;
context: string;
score: number;
}
async function searchNotes(
query: string,
notesDir: string
): Promise<SearchResult[]> {
const results: SearchResult[] = [];
const queryTerms = query.toLowerCase().split(/\s+/);
const files = await readdir(notesDir, { recursive: true });
for (const file of files) {
if (!file.endsWith('.md')) continue;
const filePath = join(notesDir, file);
const content = await readFile(filePath, 'utf-8');
const lines = content.split('\n');
for (let i = 0; i < lines.length; i++) {
const lower = lines[i].toLowerCase();
const matchCount = queryTerms.filter(t => lower.includes(t)).length;
if (matchCount > 0) {
const start = Math.max(0, i - 1);
const end = Math.min(lines.length, i + 2);
results.push({
file,
line: i + 1,
context: lines.slice(start, end).join('\n'),
score: matchCount / queryTerms.length,
});
}
}
}
return results
.sort((a, b) => b.score - a.score)
.slice(0, 20);
}
// Usage: search("connection pool timeout postgres")
// ❌ Searching only by filename
const result = notes.filter(n => n.filename.includes(query));
// Misses notes where the content matches but the filename doesn't
// ✅ Multi-signal search: filename + content + tags + links
function searchScore(note: Note, query: string): number {
const terms = query.toLowerCase().split(/\s+/);
let score = 0;
for (const term of terms) {
if (note.filename.toLowerCase().includes(term)) score += 3;
if (note.tags.some(t => t.includes(term))) score += 2;
if (note.content.toLowerCase().includes(term)) score += 1;
if (note.links.some(l => l.toLowerCase().includes(term))) score += 1;
}
return score;
}The best knowledge base is the one you actually search. If you find yourself Googling the same problem you solved six months ago, your retrieval system is failing.
Spaced Review
New notes need reinforcement. A weekly review habit prevents knowledge from atrophying.
## Weekly Review Template (templates/weekly-review.md)
### Date: 2021-06-25
#### New Notes This Week (review for accuracy and links)
- [ ] Statement timeout strategy — linked to connection pooling?
- [ ] CSS aspect-ratio discovery — linked to performance MOC?
- [ ] Production incident postmortem — root cause documented?
#### Random Resurfacing (revisit 3 random older notes)
- [ ] [[distributed-tracing-fundamentals]] — still accurate?
- [ ] [[feature-flags-at-scale]] — any new patterns learned since?
- [ ] [[oauth2-flows-demystified]] — relevant to current project?
#### MOC Updates
- [ ] Database Performance MOC — add new connection pool notes
- [ ] Production Incidents MOC — add this week's incident
#### Gaps Identified
- Need deeper notes on pgBouncer configuration
- Missing: comparison of connection pooling libraries for Node.jsThe weekly review does three things: reinforces new knowledge, resurfaces old knowledge, and identifies gaps. The "random resurfacing" section prevents recent notes from crowding out older ones.
File System Structure
Keep the file structure simple. Complexity in organization creates friction that kills the habit.
knowledge-base/
├── daily/ # Quick captures, inbox
│ ├── 2021-06-23.md
│ ├── 2021-06-24.md
│ └── 2021-06-25.md
├── notes/ # Processed, permanent notes
│ ├── connection-pooling-essentials.md
│ ├── statement-timeout-strategy.md
│ └── css-aspect-ratio-layout-shift.md
├── mocs/ # Maps of Content (index notes)
│ ├── database-performance.md
│ ├── frontend-performance.md
│ └── production-incidents.md
├── projects/ # Project-specific knowledge
│ ├── migration-to-k8s/
│ └── auth-service-redesign/
├── templates/ # Capture and review templates
│ ├── daily-note.md
│ ├── weekly-review.md
│ └── incident-postmortem.md
└── README.md # How this system works
Three folders handle 90% of the workflow: daily for raw captures, notes for processed knowledge, and mocs for navigation. Everything else is optional.
Key Takeaways
- Capture immediately — write it down when you learn it, not later when the context is gone
- Use Maps of Content to organize notes into navigable clusters instead of rigid folder hierarchies
- Link aggressively — every note should connect to at least 2-3 related notes
- Build retrieval into your workflow — if you cannot find it in 30 seconds, improve your search
- Review weekly — reinforce new notes, resurface old ones, and identify knowledge gaps
- Keep the structure minimal — three folders (daily, notes, MOCs) handle most workflows
