Building a URL Shortener from Scratch

A step-by-step tutorial for a URL shortener with short code generation, redirect handling, click analytics and rate limiting — schema through API.

Published on May 25, 2022Updated on September 9, 20265 min read

tutorials url-shortener system-design nodejs typescript api-design

A URL shortener is one of the best system design exercises because it looks simple — take a long URL, return a short one — but the implementation touches hashing, database design, caching, analytics, and rate limiting. Building one from scratch teaches you more about backend engineering than most tutorials.

We will build a URL shortener that generates short codes, handles redirects, tracks click analytics, and enforces rate limits. The stack is Node.js with TypeScript, but the patterns apply to any language.

Database Schema

The core data model is straightforward: a mapping from short code to original URL, plus metadata for analytics and expiration.

sqlsql

-- Core URL mapping table
CREATE TABLE urls (
  id            BIGSERIAL PRIMARY KEY,
  short_code    VARCHAR(10) UNIQUE NOT NULL,
  original_url  TEXT NOT NULL,
  created_at    TIMESTAMPTZ DEFAULT NOW(),
  expires_at    TIMESTAMPTZ,
  click_count   BIGINT DEFAULT 0,
  creator_ip    INET,
  is_active     BOOLEAN DEFAULT TRUE
);
 
-- Index for fast lookups by short code (the hot path)
CREATE INDEX idx_urls_short_code ON urls (short_code)
  WHERE is_active = TRUE;
 
-- Click analytics table
CREATE TABLE clicks (
  id          BIGSERIAL PRIMARY KEY,
  url_id      BIGINT REFERENCES urls(id),
  clicked_at  TIMESTAMPTZ DEFAULT NOW(),
  referrer    TEXT,
  user_agent  TEXT,
  country     VARCHAR(2)
);
 
-- Index for analytics queries
CREATE INDEX idx_clicks_url_id ON clicks (url_id);
CREATE INDEX idx_clicks_clicked_at ON clicks (clicked_at);

Short Code Generation

The short code is the core of the system. It needs to be short (6-8 characters), unique, and URL-safe. There are two main approaches: counter-based encoding and random generation.

tstypescript

import crypto from 'crypto';
 
const BASE62_CHARS =
  '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz';
 
// Approach 1: Counter-based (deterministic, no collisions)
function encodeBase62(num: number): string {
  if (num === 0) return BASE62_CHARS[0];
 
  let result = '';
  let value = num;
 
  while (value > 0) {
    result = BASE62_CHARS[value % 62] + result;
    value = Math.floor(value / 62);
  }
 
  return result;
}
 
// Approach 2: Random generation (needs collision check)
function generateRandomCode(length: number = 7): string {
  const bytes = crypto.randomBytes(length);
  let result = '';
 
  for (let i = 0; i < length; i++) {
    result += BASE62_CHARS[bytes[i] % 62];
  }
 
  return result;
}

tstypescript

// ❌ Using MD5/SHA hash of the URL — wasteful and collision-prone
function badShortCode(url: string): string {
  const hash = crypto.createHash('md5').update(url).digest('hex');
  return hash.substring(0, 7); // Truncating a hash increases collision risk
}
 
// ✅ Using counter-based encoding — guaranteed unique, predictable
async function generateShortCode(db: Database): Promise<string> {
  // Use database sequence for guaranteed uniqueness
  const result = await db.query(
    "SELECT nextval('url_id_seq') AS id"
  );
  const id = Number(result.rows[0].id);
 
  // Offset to avoid short codes like "0", "1", "a"
  // Starting at 100_000_000 gives 7-character codes
  return encodeBase62(id + 100_000_000);
}

URL Creation Endpoint

The creation endpoint validates the URL, generates a short code, stores the mapping, and returns the shortened URL.

tstypescript

import { z } from 'zod';
 
const CreateUrlSchema = z.object({
  url: z.string().url().max(2048),
  expiresIn: z.number().min(3600).max(31536000).optional(), // 1h to 1yr in seconds
  customCode: z
    .string()
    .regex(/^[a-zA-Z0-9_-]+$/)
    .min(4)
    .max(20)
    .optional(),
});
 
async function createShortUrl(
  req: Request,
  db: Database,
  config: AppConfig
): Promise<Response> {
  const body = CreateUrlSchema.parse(await req.json());
 
  // Check if custom code is already taken
  if (body.customCode) {
    const existing = await db.query(
      'SELECT id FROM urls WHERE short_code = $1',
      [body.customCode]
    );
    if (existing.rows.length > 0) {
      return new Response(
        JSON.stringify({ error: 'Custom code already in use' }),
        { status: 409 }
      );
    }
  }
 
  const shortCode = body.customCode ?? await generateShortCode(db);
  const expiresAt = body.expiresIn
    ? new Date(Date.now() + body.expiresIn * 1000)
    : null;
 
  await db.query(
    `INSERT INTO urls (short_code, original_url, expires_at, creator_ip)
     VALUES ($1, $2, $3, $4)`,
    [shortCode, body.url, expiresAt, getClientIp(req)]
  );
 
  return new Response(
    JSON.stringify({
      shortUrl: `${config.baseUrl}/${shortCode}`,
      shortCode,
      expiresAt,
    }),
    { status: 201 }
  );
}

Redirect Handler

The redirect endpoint is the hot path — every click hits this. It must be fast. Look up the short code, record the click asynchronously, and redirect.

tstypescript

async function handleRedirect(
  req: Request,
  shortCode: string,
  db: Database,
  cache: RedisClient
): Promise<Response> {
  // Check cache first (most short URLs are accessed repeatedly)
  let originalUrl = await cache.get(`url:${shortCode}`);
 
  if (!originalUrl) {
    // Cache miss — query the database
    const result = await db.query(
      `SELECT original_url, expires_at, is_active FROM urls
       WHERE short_code = $1`,
      [shortCode]
    );
 
    if (result.rows.length === 0) {
      return new Response('Not Found', { status: 404 });
    }
 
    const row = result.rows[0];
 
    if (!row.is_active) {
      return new Response('This link has been deactivated', { status: 410 });
    }
 
    if (row.expires_at && new Date(row.expires_at) < new Date()) {
      return new Response('This link has expired', { status: 410 });
    }
 
    originalUrl = row.original_url;
 
    // Cache for 1 hour
    await cache.set(`url:${shortCode}`, originalUrl, 'EX', 3600);
  }
 
  // Record click asynchronously — do not block the redirect
  recordClick(shortCode, req, db).catch((err) =>
    console.error('Failed to record click:', err)
  );
 
  return new Response(null, {
    status: 302,
    headers: { Location: originalUrl },
  });
}
 
async function recordClick(
  shortCode: string,
  req: Request,
  db: Database
): Promise<void> {
  const userAgent = req.headers.get('user-agent') ?? '';
  const referrer = req.headers.get('referer') ?? '';
 
  await db.query(
    `INSERT INTO clicks (url_id, user_agent, referrer)
     SELECT id, $2, $3 FROM urls WHERE short_code = $1`,
    [shortCode, userAgent, referrer]
  );
 
  // Increment the denormalized counter
  await db.query(
    'UPDATE urls SET click_count = click_count + 1 WHERE short_code = $1',
    [shortCode]
  );
}

Rate Limiting

Without rate limiting, anyone can flood the creation endpoint and exhaust your short code space. A sliding window rate limiter using Redis is simple and effective.

tstypescript

async function checkRateLimit(
  clientIp: string,
  cache: RedisClient,
  config: { maxRequests: number; windowSeconds: number }
): Promise<{ allowed: boolean; remaining: number; resetAt: number }> {
  const key = `ratelimit:${clientIp}`;
  const now = Date.now();
  const windowStart = now - config.windowSeconds * 1000;
 
  // Use a sorted set with timestamps as scores
  const pipeline = cache.pipeline();
  pipeline.zremrangebyscore(key, 0, windowStart); // Remove old entries
  pipeline.zadd(key, now.toString(), `${now}:${Math.random()}`);
  pipeline.zcard(key); // Count entries in window
  pipeline.expire(key, config.windowSeconds); // Auto-cleanup
 
  const results = await pipeline.exec();
  const requestCount = results?.[2]?.[1] as number;
 
  return {
    allowed: requestCount <= config.maxRequests,
    remaining: Math.max(0, config.maxRequests - requestCount),
    resetAt: Math.ceil((windowStart + config.windowSeconds * 1000) / 1000),
  };
}

tstypescript

// ❌ No rate limiting — vulnerable to abuse
app.post('/api/shorten', async (req, res) => {
  const result = await createShortUrl(req, db, config);
  return res.json(result);
});
 
// ✅ Rate limited — 100 URLs per hour per IP
app.post('/api/shorten', async (req, res) => {
  const clientIp = getClientIp(req);
  const limit = await checkRateLimit(clientIp, redis, {
    maxRequests: 100,
    windowSeconds: 3600,
  });
 
  if (!limit.allowed) {
    res.set('Retry-After', String(limit.resetAt - Math.floor(Date.now() / 1000)));
    return res.status(429).json({
      error: 'Rate limit exceeded',
      retryAfter: limit.resetAt,
    });
  }
 
  res.set('X-RateLimit-Remaining', String(limit.remaining));
  const result = await createShortUrl(req, db, config);
  return res.json(result);
});

Click Analytics Endpoint

Basic analytics give users insight into how their links are performing. Aggregate data by time period to keep queries fast.

tstypescript

async function getUrlAnalytics(
  shortCode: string,
  db: Database
): Promise<Analytics> {
  const [urlResult, dailyClicks, topReferrers] = await Promise.all([
    db.query(
      'SELECT click_count, created_at FROM urls WHERE short_code = $1',
      [shortCode]
    ),
    db.query(
      `SELECT DATE(clicked_at) AS day, COUNT(*) AS clicks
       FROM clicks
       JOIN urls ON urls.id = clicks.url_id
       WHERE urls.short_code = $1
         AND clicked_at > NOW() - INTERVAL '30 days'
       GROUP BY DATE(clicked_at)
       ORDER BY day DESC`,
      [shortCode]
    ),
    db.query(
      `SELECT referrer, COUNT(*) AS clicks
       FROM clicks
       JOIN urls ON urls.id = clicks.url_id
       WHERE urls.short_code = $1
         AND referrer != ''
       GROUP BY referrer
       ORDER BY clicks DESC
       LIMIT 10`,
      [shortCode]
    ),
  ]);
 
  return {
    totalClicks: urlResult.rows[0]?.click_count ?? 0,
    createdAt: urlResult.rows[0]?.created_at,
    dailyClicks: dailyClicks.rows,
    topReferrers: topReferrers.rows,
  };
}

Key Takeaways

Counter-based short codes are simpler than random — use a database sequence to guarantee uniqueness without collision handling
Cache the redirect path aggressively — this is the hottest path in the system; every millisecond matters
Record analytics asynchronously — never block a redirect to write a click record
Rate limit the creation endpoint — without limits, your short code space and database are vulnerable to abuse
Validate and sanitize URLs — reject obviously malicious or malformed input at the boundary
Use parameterized queries for all database operations — URL shorteners handle user-provided URLs that could contain anything

Building a URL Shortener from Scratch

Database Schema

Short Code Generation

URL Creation Endpoint

Redirect Handler

Rate Limiting

Click Analytics Endpoint

Key Takeaways

Related Posts

Building a GraphQL Server From Scratch

Building a CLI Tool with Node.js from Scratch

Building a Rate Limiter From Scratch