Building a Markdown Parser from Scratch in TypeScript

Build a Markdown parser step by step — lexical analysis, AST construction and HTML rendering — to understand how text processing engines work inside.

Published on April 6, 2023Updated on September 9, 20265 min read

typescript parsing markdown compiler-design ast tutorial

Parsers are everywhere—in compilers, template engines, configuration files, and the Markdown renderer displaying this very post. Building one from scratch teaches you how structured text becomes structured data, a skill that transfers to every text processing problem you encounter.

We'll build a subset Markdown parser that handles headings, paragraphs, bold, italic, code blocks, and inline code. The architecture follows the classic pipeline: raw text → tokens → abstract syntax tree → HTML output.

Defining Token Types

The lexer breaks raw Markdown text into meaningful tokens. Each token carries a type and the raw content it represents.

tstypescript

type TokenType =
  | "heading"
  | "paragraph"
  | "text"
  | "bold"
  | "italic"
  | "code-block"
  | "inline-code"
  | "newline"
  | "eof";
 
interface Token {
  type: TokenType;
  value: string;
  level?: number; // For headings: 1-6
  language?: string; // For code blocks
}

Building the Lexer

The lexer scans through the input string and produces tokens. It processes the document line by line for block-level elements, then handles inline formatting within text content.

tstypescript

// ❌ Naive approach: regex-replace everything at once
function badParse(markdown: string): string {
  return markdown
    .replace(/^### (.+)$/gm, "<h3>$1</h3>")
    .replace(/\*\*(.+?)\*\*/g, "<strong>$1</strong>")
    .replace(/\*(.+?)\*/g, "<em>$1</em>");
  // Breaks on nested formatting, multi-line elements,
  // code blocks containing markdown syntax, etc.
}

tstypescript

// ✅ Proper lexer with state tracking
class Lexer {
  private input: string;
  private pos: number = 0;
  private tokens: Token[] = [];
 
  constructor(input: string) {
    this.input = input;
  }
 
  tokenize(): Token[] {
    const lines = this.input.split("\n");
    let i = 0;
 
    while (i < lines.length) {
      const line = lines[i];
 
      // Code block (fenced)
      if (line.startsWith("```")) {
        const language = line.slice(3).trim();
        const codeLines: string[] = [];
        i++;
 
        while (i < lines.length && !lines[i].startsWith("```")) {
          codeLines.push(lines[i]);
          i++;
        }
 
        this.tokens.push({
          type: "code-block",
          value: codeLines.join("\n"),
          language: language || undefined,
        });
        i++; // Skip closing ```
        continue;
      }
 
      // Heading
      const headingMatch = line.match(/^(#{2,6})\s+(.+)$/);
      if (headingMatch) {
        this.tokens.push({
          type: "heading",
          value: headingMatch[2],
          level: headingMatch[1].length,
        });
        i++;
        continue;
      }
 
      // Empty line
      if (line.trim() === "") {
        this.tokens.push({ type: "newline", value: "\n" });
        i++;
        continue;
      }
 
      // Paragraph text — collect consecutive non-empty lines
      const paragraphLines: string[] = [];
      while (
        i < lines.length &&
        lines[i].trim() !== "" &&
        !lines[i].startsWith("```") &&
        !lines[i].match(/^#{2,6}\s/)
      ) {
        paragraphLines.push(lines[i]);
        i++;
      }
 
      if (paragraphLines.length > 0) {
        this.tokens.push({
          type: "paragraph",
          value: paragraphLines.join(" "),
        });
      }
    }
 
    this.tokens.push({ type: "eof", value: "" });
    return this.tokens;
  }
}

The lexer handles code blocks as opaque content—everything between the fences is captured as-is without trying to parse Markdown syntax inside it.

The Abstract Syntax Tree

The AST gives us a structured representation that's easy to transform into any output format.

tstypescript

type ASTNode =
  | HeadingNode
  | ParagraphNode
  | TextNode
  | BoldNode
  | ItalicNode
  | CodeBlockNode
  | InlineCodeNode
  | DocumentNode;
 
interface DocumentNode {
  type: "document";
  children: ASTNode[];
}
 
interface HeadingNode {
  type: "heading";
  level: number;
  children: ASTNode[];
}
 
interface ParagraphNode {
  type: "paragraph";
  children: ASTNode[];
}
 
interface TextNode {
  type: "text";
  value: string;
}
 
interface BoldNode {
  type: "bold";
  children: ASTNode[];
}
 
interface ItalicNode {
  type: "italic";
  children: ASTNode[];
}
 
interface CodeBlockNode {
  type: "code-block";
  language?: string;
  value: string;
}
 
interface InlineCodeNode {
  type: "inline-code";
  value: string;
}

Parsing Inline Content

Inline parsing handles bold, italic, and inline code within text. This is the trickiest part because these formats can nest.

tstypescript

class InlineParser {
  private input: string;
  private pos: number = 0;
 
  constructor(input: string) {
    this.input = input;
  }
 
  parse(): ASTNode[] {
    const nodes: ASTNode[] = [];
    let textBuffer = "";
 
    while (this.pos < this.input.length) {
      const char = this.input[this.pos];
      const next = this.input[this.pos + 1];
 
      // Inline code
      if (char === "`") {
        if (textBuffer) {
          nodes.push({ type: "text", value: textBuffer });
          textBuffer = "";
        }
        nodes.push(this.parseInlineCode());
        continue;
      }
 
      // Bold: **text**
      if (char === "*" && next === "*") {
        if (textBuffer) {
          nodes.push({ type: "text", value: textBuffer });
          textBuffer = "";
        }
        nodes.push(this.parseBold());
        continue;
      }
 
      // Italic: *text*
      if (char === "*" && next !== "*") {
        if (textBuffer) {
          nodes.push({ type: "text", value: textBuffer });
          textBuffer = "";
        }
        nodes.push(this.parseItalic());
        continue;
      }
 
      textBuffer += char;
      this.pos++;
    }
 
    if (textBuffer) {
      nodes.push({ type: "text", value: textBuffer });
    }
 
    return nodes;
  }
 
  private parseInlineCode(): InlineCodeNode {
    this.pos++; // Skip opening `
    let value = "";
 
    while (this.pos < this.input.length && this.input[this.pos] !== "`") {
      value += this.input[this.pos];
      this.pos++;
    }
 
    this.pos++; // Skip closing `
    return { type: "inline-code", value };
  }
 
  private parseBold(): BoldNode {
    this.pos += 2; // Skip opening **
    let content = "";
 
    while (
      this.pos < this.input.length - 1 &&
      !(this.input[this.pos] === "*" && this.input[this.pos + 1] === "*")
    ) {
      content += this.input[this.pos];
      this.pos++;
    }
 
    this.pos += 2; // Skip closing **
    const innerParser = new InlineParser(content);
    return { type: "bold", children: innerParser.parse() };
  }
 
  private parseItalic(): ItalicNode {
    this.pos++; // Skip opening *
    let content = "";
 
    while (this.pos < this.input.length && this.input[this.pos] !== "*") {
      content += this.input[this.pos];
      this.pos++;
    }
 
    this.pos++; // Skip closing *
    const innerParser = new InlineParser(content);
    return { type: "italic", children: innerParser.parse() };
  }
}

Note how parseBold recursively creates a new InlineParser for its content. This allows nested formatting like **bold with *italic* inside** to parse correctly.

Building the Document Parser

The document parser takes the token stream and builds the full AST by combining block-level tokens with inline parsing.

tstypescript

class Parser {
  private tokens: Token[];
  private pos: number = 0;
 
  constructor(tokens: Token[]) {
    this.tokens = tokens;
  }
 
  parse(): DocumentNode {
    const children: ASTNode[] = [];
 
    while (this.current().type !== "eof") {
      const token = this.current();
 
      if (token.type === "newline") {
        this.pos++;
        continue;
      }
 
      if (token.type === "heading") {
        const inlineParser = new InlineParser(token.value);
        children.push({
          type: "heading",
          level: token.level!,
          children: inlineParser.parse(),
        });
        this.pos++;
        continue;
      }
 
      if (token.type === "code-block") {
        children.push({
          type: "code-block",
          language: token.language,
          value: token.value,
        });
        this.pos++;
        continue;
      }
 
      if (token.type === "paragraph") {
        const inlineParser = new InlineParser(token.value);
        children.push({
          type: "paragraph",
          children: inlineParser.parse(),
        });
        this.pos++;
        continue;
      }
 
      this.pos++;
    }
 
    return { type: "document", children };
  }
 
  private current(): Token {
    return this.tokens[this.pos] ?? { type: "eof", value: "" };
  }
}

Rendering to HTML

The renderer walks the AST and produces HTML. Because the tree is well-structured, rendering is a straightforward recursive traversal.

tstypescript

function escapeHtml(text: string): string {
  return text
    .replace(/&/g, "&amp;")
    .replace(/</g, "&lt;")
    .replace(/>/g, "&gt;")
    .replace(/"/g, "&quot;");
}
 
function renderNode(node: ASTNode): string {
  switch (node.type) {
    case "document":
      return node.children.map(renderNode).join("\n");
 
    case "heading":
      const tag = `h${node.level}`;
      const headingContent = node.children.map(renderNode).join("");
      return `<${tag}>${headingContent}</${tag}>`;
 
    case "paragraph":
      const paraContent = node.children.map(renderNode).join("");
      return `<p>${paraContent}</p>`;
 
    case "text":
      return escapeHtml(node.value);
 
    case "bold":
      const boldContent = node.children.map(renderNode).join("");
      return `<strong>${boldContent}</strong>`;
 
    case "italic":
      const italicContent = node.children.map(renderNode).join("");
      return `<em>${italicContent}</em>`;
 
    case "code-block":
      const langAttr = node.language
        ? ` class="language-${escapeHtml(node.language)}"`
        : "";
      return `<pre><code${langAttr}>${escapeHtml(node.value)}</code></pre>`;
 
    case "inline-code":
      return `<code>${escapeHtml(node.value)}</code>`;
 
    default:
      return "";
  }
}

The escapeHtml function is critical—without it, user-generated Markdown containing <script> tags would produce executable HTML.

Putting It All Together

tstypescript

function parseMarkdown(input: string): string {
  const lexer = new Lexer(input);
  const tokens = lexer.tokenize();
 
  const parser = new Parser(tokens);
  const ast = parser.parse();
 
  return renderNode(ast);
}
 
// Usage
const markdown = `## Hello World
 
This is a **bold** and *italic* paragraph with \`inline code\`.
 
\`\`\`typescript
const x = 42;
console.log(x);
\`\`\`
 
Another paragraph with **nested *formatting* here**.`;
 
console.log(parseMarkdown(markdown));

Key Takeaways

Building a parser from scratch reinforces a fundamental pattern that appears everywhere in software: raw input → structured tokens → abstract tree → output format. The same pipeline powers compilers, template engines, query languages, and configuration parsers. The specific techniques—state-tracking lexers, recursive descent parsing, AST-based rendering—are tools you'll reach for whenever you need to process structured text. Start simple, get the pipeline working end to end, then add features incrementally. A parser that handles five constructs correctly is more valuable than one that handles twenty constructs with subtle bugs.

Building a Markdown Parser from Scratch in TypeScript

Defining Token Types

Building the Lexer

The Abstract Syntax Tree

Parsing Inline Content

Building the Document Parser

Rendering to HTML

Putting It All Together

Key Takeaways

Related Posts

Building a Real-Time Notification System from Scratch

Building a Full-Text Search Engine from Scratch

Migrating from JavaScript to TypeScript Incrementally