Interactive Series · 10 Parts

Compilers —
From Source Text to Running Code

A ten-part, deeply illustrated tour of how a compiler turns the text you type into instructions a machine runs — the history, the classic pipeline, and every stage in between, ending by building a small compiler end to end.

source→ tokens→ AST→ typed AST→ IR / SSA→ opt IR→ asm→ machine code
Lexing Finite Automata CFGs LL / LR / LALR Type Inference Hindley–Milner SSA LLVM IR Data-Flow Analysis Register Allocation JIT GC WebAssembly

Who this is for. Programmers who want to understand what happens between “save” and “run” — CS students, language hobbyists, and engineers who use compilers daily and want to see inside one. No prior compiler experience assumed; comfort reading code helps.

How to use it. Each part is a self-contained interactive deck (Reveal.js) with hand-drawn diagrams and copy-and-run code. Read them in order for a full course, or jump to a topic. Every deck also ships a plain-Markdown version for reading on GitHub.

The compiler pipeline at a glance

Front End · analysis Middle End Back End · synthesis Lexer03 Parser04 Semantic05 IR / SSA06 Optimiser07 Codegen08 Runtime / JIT09 source text machine code Symbol Table · Error Reporting · Source Locations — threaded through every phase History & framing in Part 01–02 · the whole thing built by hand in Part 10

Presentations in this series

  1. 01
    A History of Compilers →
    Seventy years from machine code to MLIR — the first assemblers (Booth, Wheeler), Grace Hopper's A-0 and the word “compiler”, FORTRAN's optimiser, ALGOL & BNF, the Chomsky hierarchy, LR/LALR parsing, lex & yacc, the Dragon Book, GCC, SSA, the JIT lineage (Self → HotSpot), LLVM, and the modern infrastructure era.
    livePart 01 · Historymarkdown
  2. 02
    The Compiler Pipeline →
    The architecture that frames the series — compiler vs interpreter vs JIT, analysis vs synthesis, the phases end to end, a worked trace of one statement through every stage, front/middle/back ends and the N×M “narrow waist”, the full toolchain (driver → linker → loader), bootstrapping & T-diagrams, and cross-compilation.
    livePart 02 · Overviewmarkdown
  3. 03
    Lexical Analysis →
    Turning characters into tokens — regular expressions and regular languages, NFA vs DFA, Thompson's construction, the subset construction and DFA minimisation, maximal munch and lookahead, hand-written vs generated scanners (lex/flex/re2c), literals and Unicode, and the hard cases (Python INDENT/DEDENT, the C lexer hack).
    livePart 03 · Front Endmarkdown
  4. 04
    Syntax Analysis (Parsing) →
    From tokens to trees — context-free grammars, derivations, parse trees vs ASTs, ambiguity and precedence; top-down (recursive descent, FIRST/FOLLOW, LL(1), Pratt parsing); bottom-up (shift-reduce, LR(0) items, SLR/LALR(1)); conflicts; the generator landscape (yacc/bison, ANTLR, tree-sitter, PEG); and error recovery.
    livePart 04 · Front Endmarkdown
  5. 05
    Semantic Analysis & Type Systems →
    Meaning, not just shape — symbol tables and scope, name resolution and shadowing, attribute grammars and syntax-directed translation, type-system axes, typing judgements (Γ ⊢ e : τ), Hindley–Milner inference and unification, subtyping & variance, generics and type classes, and producing good diagnostics.
    livePart 05 · Front Endmarkdown
  6. 06
    Intermediate Representations →
    The language the middle end speaks — why an IR, the HIR/MIR/LIR spectrum, three-address code, stack vs register bytecode (JVM/CPython/Wasm vs Lua/Dalvik), control-flow graphs, dominators, SSA and φ-functions (construction and destruction), LLVM IR in depth, plus CPS, sea-of-nodes and MLIR.
    livePart 06 · Middle Endmarkdown
  7. 07
    Optimization →
    Making it faster without changing what it does — the “as-if” rule, scope levels up to LTO & PGO, the data-flow framework (lattices, transfer functions, fixed points), the four classic analyses, the classic transforms (CSE, DCE, inlining, strength reduction), loop optimisations & vectorisation, SSA-based passes (SCCP, GVN), alias analysis and phase ordering.
    livePart 07 · Middle Endmarkdown
  8. 08
    Code Generation →
    IR down to real instructions — the three interacting subproblems; instruction selection (tree tiling, BURS, SelectionDAG/GlobalISel); register allocation (interference graphs, graph colouring, Chaitin–Briggs, linear scan, coalescing); instruction scheduling and pipeline hazards; the System V AMD64 ABI, stack frames and prologue/epilogue; a worked x86-64 / AArch64 example.
    livePart 08 · Back Endmarkdown
  9. 09
    Runtimes, JITs & Modern Backends →
    What runs the code — interpreters and dispatch (threaded code, computed goto), JITs (baseline vs optimising, tracing, tiered execution in V8/HotSpot), inline caches, deoptimisation and OSR, garbage collection (generational, concurrent), exception handling and dynamic linking (PLT/GOT), and modern backends: LLVM, Cranelift, GraalVM, MLIR and WebAssembly.
    livePart 09 · Runtimemarkdown
  10. 10
    Building a Mini-Compiler →
    The capstone — one tiny language (“MiniLang”) threaded through the whole pipeline in real, runnable code: grammar → hand-written lexer → recursive-descent + Pratt parser → type checker → three-address IR → constant folding & DCE → stack-VM bytecode → an interpreter that runs it — plus testing, error messages, and where to go next.
    livePart 10 · Capstonemarkdown

The learning arc

Frame it

Parts 01–02 — the history of the field and the pipeline that organises everything that follows.

Front end

Parts 03–05 — lexing, parsing and semantic analysis: text → tokens → a typed tree.

Middle end

Parts 06–07 — the IR (with SSA) and the optimiser that improves it, target-independently.

Back end

Part 08 — code generation: instruction selection, register allocation and scheduling down to assembly.

Run it

Part 09 — runtimes, JITs and GC; how modern engines and backends (LLVM, Wasm) actually execute code.

Build it

Part 10 — the capstone: assemble everything into a working compiler for a small language.

Navigating a deck

Next / Previous — → ← or swipe
Slide overview — Esc
Fullscreen — F
Export to PDF — append ?print-pdf then print

How these are built

Every deck is a single self-contained index.html — no build step, no npm install, nothing to download. Slides are authored in HTML and rendered with Reveal.js 4.6; code is highlighted with highlight.js; every diagram is hand-drawn inline SVG. Typeset in Schibsted Grotesk and Azeret Mono. Each deck also ships a plain-Markdown presentation.md for reading on GitHub.

References

Aho, Lam, Sethi & Ullman, Compilers: Principles, Techniques & Tools (the “Dragon Book”) · Robert Nystrom, Crafting Interpreters · Appel, Modern Compiler Implementation · Cooper & Torczon, Engineering a Compiler · LLVM documentation · Kaleidoscope tutorial · WebAssembly