LLMs

Brendan Lynskey · Portfolio home · Interactive sites

LLMs

A hierarchical index of presentation series for AI / LLM / agentic engineers — from transformer internals and GPU hardware up through retrieval, agents, evaluations, safety and production operations.

Every project that used to be indexed here directly is still reachable — typically through its sub-hub (one click of indirection). The 26 sub-hubs below collectively re-export ~209 leaf repos: ~80 pre-existing presentations, 5 MCP decks, 78 decks across 14 mid-2025 series (now including a two-deck deep-dive companion on the two-step retrieval cascade architecture and the underlying reranker mathematics), the 10-deck LLM History series covering people, labs and ideas from Shannon to the multipolar 2026 frontier, the 12-deck Mathematics for Machine Learning companion to Deisenroth, Faisal & Ong, and the 7-deck Karpathy: Neural Networks Zero to Hero series walking through Andrej Karpathy’s from-scratch teaching codebases (micrograd, makemore, minbpe, minGPT, nanoGPT, llm.c, nanochat), and the 10-deck Key LLM Publications series indexing the fifty most important papers, articles and blog posts for a practising LLM/agent engineer — from the Transformer to the Model Context Protocol — each with a summary, an engineer’s interpretation, and diagrams, and the 6-deck Transformer Circuits Thread series presenting Anthropic’s mechanistic-interpretability research from transformer-circuits.pub (a mathematical framework for transformer circuits, induction heads, superposition, dictionary-learning features, their scaling to Claude 3 Sonnet, and circuit tracing) — plus the directly-linked code repos and articles below.

Start here: the interactive sites

Sixteen companion explainer sites in four tracks, each a computed, animated reading experience whose numbers come from tested code (every site’s About page says how it is checked). Each site’s header has a menu to all the others. They are also open source: the repositories ship the models, fixtures and CI.

Foundations: one small model, end to end

Site Repo Sub-hub What’s inside
Transformer Decoder Explained: how a decoder-only transformer runs, one forward pass · Transformer Training Explained: how the same decoder is trained, from one step to many GPUs transformer-explainer Transformer Architecture Foundations › Decoder (site menu). Full-stack interactive web app at transformer-decoder-explained.vercel.app — type your own tokens and watch every op (embeddings, masked self-attention, FFN, layernorm, sampling) execute server-side (the Decoder site). Training (Foundations › Training), at its own address: 16 chapters that train the same model in your browser, from backprop and AdamW to mixed precision, parallelism, training over slow links and scaling laws; slides in the Transformer Training hub. Pure-TypeScript transformer library verified against PyTorch fixtures to 1e-5. Ships with a backend-engineering PDF tour and a Reveal.js presentation walking through Next.js 14, Drizzle, Auth.js, the test pyramid, CI/CD, and the six production bugs we hit going live

Hardware: the machines underneath, chip to rack

Site Repo Sub-hub What’s inside
GPU Kernels Explained: how a GPU executes the maths gpu-kernels-explained NVIDIA GPU Architectures Hardware › Kernels (site menu). Fourth companion interactive web app at gpu-kernels-explained.vercel.app — how a GPU actually executes the maths, in 11 animated chapters: the memory hierarchy, the roofline, warps and divergence, coalescing, shared-memory bank conflicts, occupancy, GEMM from naive to tensor cores, reductions and warp shuffles, online softmax and FlashAttention, split-K and overlap, quantised kernels. Every frame of every animation is computed by a parameterised GPU execution model (A100 and H100 presets, every figure sourced to NVIDIA’s whitepapers, the CUDA Programming Guide or published microbenchmarks) whose TypeScript port matches its Python reference exactly. Links the matching slides of the NVIDIA GPU and CUDA series from every chapter. Same design system, linked from the other sites’ headers
Systolic Arrays Explained: inside a matrix engine, checked against SystemVerilog in Verilator systolic-arrays-explained Google TPUs Hardware › Systolic (site menu). Sixth companion interactive web app at systolic-arrays-explained.vercel.app — the matrix hardware under the models, in 9 animated chapters: why systolic (the memory wall), weight-stationary cycle by cycle, output- and input-stationary, skew, fill and drain, tiling big GEMMs with double-buffered weights, inside a PE (bfloat16/FP16/INT8 MAC pipeline), the TPU and an all-reduce on a 2-D torus, other ways to build it (row-stationary, tensor cores), and an ONNX graph lowered onto the array. Every frame is a cycle of a cycle-accurate model (Python reference, exact TypeScript port) checked against the author’s SystemVerilog systolic array and MAC unit in Verilator. Links the Google TPU series. Same design system, linked from the other sites’ headers
AI Accelerators Explained: the families of AI chips on the same work ai-accelerators-explained AI Accelerators (and NVIDIA GPU Architectures, Google TPUs) Hardware › Accelerators (site menu). Companion interactive web app at ai-accelerators-explained.vercel.app — the families of AI silicon compared on the same work (a Llama decoder’s prefill and decode), in 10 animated chapters: the design space (where the weights live, who decides when data moves), GPUs as the baseline, systolic designs (TPU, Trainium), wafer-scale (yield and spares), compiler-scheduled SRAM chips (a token down a chain of LPUs), reconfigurable dataflow (a layer fused into one pipeline; three-tier memory; the IPU), meshes of small cores (Tenstorrent, Ethernet on the card), FPGAs and edge NPUs (a batch-1 decode at narrower weights), in-memory compute (a decoder on phase-change crossbars), and every family side by side. A Python accelerator model with an exact TypeScript port; every specification sourced and dated, announced, retired and research parts marked, vendor claims quoted, never inputs; a validation band against published figures. Slide companion: the AI Accelerators series
AI Racks Explained: GPUs, networks, power and cooling at scale ai-racks-explained NVIDIA GPU Architectures Hardware › Racks (site menu). Companion interactive web app at ai-racks-explained.vercel.app — the machines that train and serve large models, chip to data centre, in 11 animated chapters: the scale ladder, the package and its HBM, trays and nodes, NVLink and the NVL72 rack, scale-out fabrics (fat trees, rail-aligned SuperPOD rules), power (shelves, busbars, power smoothing), cooling (the coolant loop), disaggregated inference on one rack (Disaggregated_Inference_Sim’s engine vendored at a pinned commit and run live in a Web Worker, bit-exact with its Python package), training clusters (layout on NVLink domains, Young’s checkpoint interval), other designs (TPU pods, AMD Helios, AWS Trainium) and cost per token. A Python rack model with an exact TypeScript port; every hardware figure sourced and dated, announced parts marked, a 27-check validation band; links NVIDIA GPU decks 05/11/13/14/15/23/24/25 and InfSim 05/07

Inference: serving models at scale

Site Repo Sub-hub What’s inside
LLM Inference Explained: how real systems generate text, with a live serving simulator llm-inference-explained LLM Inference Simulators Inference › Serving (site menu). Companion interactive web app at llm-inference-explained.vercel.app — 14 chapters on real autoregressive inference: the generation loop, a KV cache proven bit-identical to recomputation on a live tiny model, the roofline, batching, paged memory, FlashAttention, speculative decoding, quantisation, parallelism, serving metrics, and disaggregated prefill/decode with Disaggregated_Inference_Sim’s engine running live in the browser, tested request by request against the Python package. Static Next.js 14, same design system as transformer-explainer, linked from its header
LLM Architectures Explained: how model designs differ, model by model llm-architectures-explained Modern Architectures Inference › Architectures (site menu). Third companion interactive web app at llm-architectures-explained.vercel.app — how 160 LLM architectures differ, from the 2017 Transformer to this year’s open and closed models (every name in Raschka’s LLM Architecture Gallery, plus landmarks), every value sourced to a pinned config, paper or model card, closed-model sizes only as labelled reported estimates. A Python cost model with an exactly-matching TypeScript port drives the model table, per-model diagrams, compare view, timeline and nine chapters (attention, positions, norms, MoE, depth/width, long context, MTP, looped/parallel blocks, encoder-decoder), and the CED chapter reruns Disaggregated_Inference_Sim’s capacity search live in the browser, rate for rate. Same design system, linked from the other two sites’ headers
Numerics Explained: number formats and quantisation numerics-explained Local LLM Hosting Inference › Numerics (site menu). Fifth companion interactive web app at numerics-explained.vercel.app — number formats, rounding and quantisation for ML, in 10 animated chapters: bits to numbers, rounding (nearest-even vs stochastic), accumulation error, the formats zoo (FP8, BF16, MX), quantisation basics, activation outliers (LLM.int8(), SmoothQuant), GPTQ, AWQ and NF4, KV-cache quantisation, quantisation in hardware. Every frame is computed by a bit-exact numerics library (Python reference checked against numpy, ml_dtypes and the OCP specs; TypeScript port matching it exactly); chapters 8–9 quantise the Transformer Decoder Explainer’s tiny model live in the browser. Links the Local LLM, Google TPU, NVIDIA GPU and Linear Algebra for AI series. Same design system, linked from the other sites’ headers
Inference Trade-offs Explained: which serving lever helps which metric inference-tradeoffs-explained LLM Inference Simulators Inference › Trade-offs (site menu). Seventh companion interactive web app at inference-tradeoffs-explained.vercel.app — architectural trade-offs in LLM serving, organised around decisions: for each lever (batching and chunked prefill, paged KV, prefix caching, disaggregation, heterogeneous and optical pools, KV hand-off and compression, CED, TP/PP/EP, quantisation, speculative decoding, power, hardware and cost) what it improves, what it costs and under which workload. A Pareto explorer, a lever-by-metric matrix and a live what-if over Disaggregated_Inference_Sim’s recorded 350-point sweep (23 levers × 5 workloads × H100/H200/B200), 13 chapters whose mechanism animations replay states recorded by the simulator itself (the site’s vendored engine reproduces their timestamps bit for bit), and five workload case studies. Same design system, linked from the other sites’ headers

Agents: building on models

Site Repo Sub-hub What’s inside
Agent Harnesses Explained: the loop between a model and the world agent-harnesses-explained Agents & Orchestration (and Coding Agents Internals) Agents › Harnesses (site menu). First of the agent companion sites, at agent-harnesses-explained.vercel.app — the harness that turns a chat model into an agent, in 10 animated chapters: the agent loop, tool calling (native calls vs ReAct text, malformed-call recovery), the context window as a budget (truncation, clipping, summarising compaction), prompt caching, permissions and the human in the loop, sub-agents, hooks and sandboxing, failure and recovery, the public coding-agent harnesses compared (a sourced matrix; the same task replayed under each one’s policy set, labelled a policy imitation, not the product), and the cost and latency of a whole task. No live model: every frame is computed by Agent_Loop_Sim with a scripted model, and three runs of a small open-weights model recorded once on a CPU are replayed token for token, failures included. Same design system; the header switch has four tracks: Foundations, Hardware, Inference and Agents
Agent Protocols Explained: MCP, transports, auth and A2A, checked against the official SDKs agent-protocols-explained Model Context Protocol (and Agents & Orchestration) Agents › Protocols (site menu). Second of the agent companion sites, at agent-protocols-explained.vercel.app — how agents talk to tools and to each other at the level of messages on the wire, in 9 animated chapters: why a protocol, JSON-RPC and the life cycle of a session, tools, resources and prompts, transports, sampling and elicitation, OAuth 2.1 authorisation, gateways and composition, agent to agent (A2A 1.0) and protocol security. MCP in both its eras (the handshake revisions 2024-11-05 to 2025-11-25, and the current 2026-07-28 revision). No live model and no real connection: every message is sent by the protocol state machines of Agent_Loop_Sim and checked against the official MCP and A2A Python SDKs, which play the same sessions in the engine’s CI. Same design system, linked from the header switch of every companion site
Agent Context Explained: retrieval, memory and context engineering, measured agent-context-explained RAG & Retrieval Systems (and Agents & Orchestration) Agents › Context (site menu). Third of the agent companion sites, at agent-context-explained.vercel.app — context engineering, measured: what goes into an agent’s context window and how an agent remembers beyond it, in 9 animated chapters: the window as working memory, lexical retrieval (BM25), dense retrieval, hybrid fusion and reranking, chunking, packing a token budget, compaction and summarisation, agent memory across sessions, and long context or retrieval. Every number is measured on a fixed, openly licensed corpus (6 SQuAD v1.1 articles, CC BY-SA 4.0, and 200 of their questions); a small embedding model and a cross-encoder ran once, offline, and their outputs ship with the site. No live model: every frame is computed by the context module of Agent_Loop_Sim, identically in Python and the browser. Same design system, linked from the header switch of every companion site
Agent Orchestration Explained: graphs, multi-agent patterns and durability, checked against LangGraph agent-orchestration-explained Agents & Orchestration Agents › Orchestration (site menu). Fourth of the agent companion sites, at agent-orchestration-explained.vercel.app — multi-step and multi-agent systems as discrete-event systems, in 9 animated chapters: from a chain to a graph (Pregel-style super-steps), state, channels and reducers, checkpoints and interrupts (crash and resume, human interrupts, time travel), durable execution (replay, non-determinism, idempotency keys), multi-agent patterns (supervisor, hierarchical, swarm, debate, map-reduce), fan-out under a rate limit, reliability with retries and verifiers, hundreds of workflows sharing one rate-limited endpoint, and choosing a pattern from the simulation results. No live model: every frame is computed by the orchestration module of Agent_Loop_Sim, whose graph runtime is checked in CI against real LangGraph 1.2.14 (39 sessions, 232 checkpoints compared after every operation). Same design system, linked from the header switch of every companion site
Agent Evals Explained: measuring agents, and how sure you can be agent-evals-explained Agents & Orchestration (and LLM Evaluations) Agents › Evals (site menu). Fifth of the agent companion sites, at agent-evals-explained.vercel.app — how to tell whether an agent is good and how sure you can be, in 10 animated chapters: what an eval is, the benchmarks from their papers (SWE-bench, τ-bench, GAIA, WebArena, OSWorld, Terminal-Bench), pass@k against pass^k, intervals and their coverage, comparing two agents (paired tests, McNemar, power), LLM judges and their biases (the swap test, Cohen’s κ, correcting a lenient judge), grading whole trajectories (including runs recorded from a small open-weights model), one simulated model under sixteen harness configurations, contamination and its detectors, and cost-aware evaluation. No live model: every frame is computed by the evals module of Agent_Loop_Sim, whose statistics are tested against scipy and statsmodels. Same design system, linked from the header switch of every companion site
Agent Security Explained: defences: trust boundaries, the lethal trifecta, least privilege agent-security-explained Agents & Orchestration (and Computer-Use & Browser Agents) Agents › Security (site menu). Sixth of the agent companion sites, at agent-security-explained.vercel.app — how to build agents that stay safe, defences only, in 10 animated chapters: trust boundaries and labels on every value, untrusted input (from the published research), the lethal trifecta checked against an exhaustive search, least privilege and audience-bound tokens, sandboxing and egress control, why layered detectors never reach zero, plan-then-execute and the dual-model pattern, approval thresholds and approval fatigue, the security–utility frontier (every defence’s cost and risk on a seeded suite of benign tasks, for fixed and adaptive attackers), and a builder’s checklist. Threats appear only as labels and cited categories, never as attack text: every frame is computed by the policy module of Agent_Loop_Sim. Same design system, linked from the header switch of every companion site

Showcases

Cloneable code repositories

Most leaves are interactive HTML presentations rendered on GitHub Pages. The repos below are different — they ship runnable source you can clone and execute outside the browser. Listed here for direct discoverability. (The interactive sites’ repositories are listed with their sites, above.)

Repo Sub-hub What’s inside
LLM_Transformer_Decoder_RTL Transformer Architecture Synthesisable SystemVerilog implementation of a pre-norm decoder block with KV-cache, plus an 83-test verification suite
Python_Transformer_Decoder Transformer Architecture Step-by-step PyTorch implementation in a single Jupyter notebook — every component built from first principles
NN_data_types Transformer Architecture SystemVerilog implementations of 9 numerical formats (FP32 down to FP4) used in NN training and inference hardware
AI_MMUL_Unit Transformer Architecture Hardware deep-dive on the MMUL — 24-slide interactive deck (Kung 1978/82 → TPU → Blackwell, four dataflows, Transformer mapping, FP32→FP4 / MXFP / NVFP4, real systems, memory hierarchy, power & thermals) plus four parameterised SystemVerilog implementations and 258 passing tests
Introduction_to_LangGraph Agents & Orchestration Interactive slide deck plus 10 runnable examples in Python and TypeScript covering graph-based agent orchestration
ReAct_math_agent Agents & Orchestration Browser-based ReAct agent that solves maths problems step-by-step with a local Ollama LLM, visualising the reasoning loop
Research_Digest_Agent Agents & Orchestration Autonomous research agent that searches the web, reads sources, and produces structured markdown digests (Claude or Ollama)
Raschka_Coding_Agent_LangGraph Coding Agents Internals LangGraph implementation of Sebastian Raschka’s Components of a Coding Agent — all six components mapped one-per-module (workspace context, cacheable stable-prefix prompt, sandboxed file/shell tools, transcript compression, two-tier memory, bounded read-only subagents), swappable Gemini / DeepSeek / OpenAI / local Qwen via Ollama / fake providers chosen in .env, MCP tool servers via langchain-mcp-adapters behind the same permission gate, full-fidelity LLM tracing (rich panels + JSONL on disk), 68 offline tests with CI on Python 3.10–3.12, written as a learning resource
Agent_Loop_Sim Agents & Orchestration Deterministic agent-harness simulator: the model → tool call → result loop with every harness choice a switchable policy (tool-calling style, permissions and a simulated human, context management, cache-aware prompt layout, sub-agents, hooks, sandbox, retries and loop detection), a vendored real tokenizer, fake tools and a versioned JSON Lines event trace. Python reference with a TypeScript port that matches its fixtures exactly; scripted, replay and local (llama.cpp) model back ends. MCP and A2A protocol state machines checked against the official Python SDKs, a context module (BM25, dense and hybrid retrieval, reranking, chunking, retrieval metrics, window packing, compaction and agent memory), and an orchestration module (a graph runtime checked against real LangGraph, checkpoints, interrupts and durable execution, multi-agent patterns, a rate-limited discrete-event simulation and reliability maths). The engine behind Agent Harnesses Explained, Agent Protocols Explained, Agent Context Explained and Agent Orchestration Explained
Disaggregated_Inference_Sim LLM Inference Simulators (and Fourier Optics for Inference) SimPy discrete-event simulator of prefill/decode-disaggregated LLM serving — roofline cost model, continuous batching, KV-capacity admission, shared KV link, colocated baseline; latency percentiles, goodput, utilisation, MFU/MBU, stage breakdown and hot-spot attribution, Perfetto traces; a power model (static + pJ/FLOP + pJ/byte + pJ/bit, DVFS, per-pool power caps, joules per token); an exact accelerated path (2.0–2.5×, bit-identical), search utilities, heterogeneous pools, FFT-mixing models, an optical transform engine and KV hand-off compression (2026-10-04), PPA, 130 tests (cost model corrected 2026-10-03 against an operator trace of the real model), and a JavaScript port that runs live in the series decks
Torch_Sim_Frontend LLM Inference Simulators (and the Simulation Engineering Toolkit) PyTorch and ONNX front ends for an accelerator cost model: a dispatch trace on the meta device or under fake tensors, torch.export, a torch.compile backend and a weightless ONNX graph walk, agreeing exactly on Llama-3-8B and checked against Disaggregated_Inference_Sim’s closed forms (it found two errors in them, since corrected) and PyTorch’s FLOP counter; traces Llama-3-70B without weights on a CPU; roofline and offload cost models; operator coverage by operators, FLOPs and time; a SimPy accelerator model (memory, DMA, buffer with back-pressure, compute array) run end to end from a graph, with stall attribution, timelines, a cycle-stepped twin, a bit-identical C++/pybind11 fast path and an NTT workload; 145 tests, GitHub Actions and Jenkins

Sub-hub index

Each row points to a sub-hub repo whose README and GitHub Pages site list the leaf decks for that area. The code marker on the Status column flags sub-hubs that include cloneable-code leaves listed above.

Foundations & Internals

Sub-hub Status What’s inside
Transformer Architecture live · 8 decks · 5 code Decoder-only transformer internals — visual walkthrough, every-computation forward pass, RTL accelerator, nanoGPT, PyTorch from scratch, NN data types, hardware-aware quantisation, MMUL hardware deep-dive, and the full-stack interactive web explainer
Karpathy: Neural Networks Zero to Hero live · 7 decks Interactive walkthroughs of Andrej Karpathy’s from-scratch teaching codebases — micrograd (autograd & backprop), makemore (char-level LMs, BatchNorm, WaveNet), minbpe (the GPT tokeniser), minGPT and nanoGPT (the GPT itself), llm.c (training in raw C/CUDA), and nanochat (a full ChatGPT pipeline). Cross-links the existing nanoGPT deck shared with the Transformer Architecture hub
Qwen live · 12 decks The complete Qwen family, generation by generation — Qwen 1.0/1.5 (tokenizer, context tricks, the first MoE), Qwen2/2.5 (GQA, 18T tokens, YaRN & dual chunk attention, Coder/Math/VL/Omni), QwQ & QVQ (RL with verifiable rewards, test-time compute), Qwen3 (hybrid thinking, 128-expert MoE, the 2507 split) with a full Qwen3-Coder deep dive (pros/cons, benchmarks, every flag, vLLM capability matrix), Qwen3-Next (Gated DeltaNet, 3:1 hybrid, MTP), the multimodal & specialist lines (VL, Omni, Embedding/Reranker, Guard, Image, ASR), Qwen3.5 (native multimodal, 201 languages), Qwen3.6→3.8 (dense returns, the 2.4T Max tier) — plus reference decks on serving Qwen with vLLM and on choosing, sizing and deploying
Modern Architectures live · 11 decks Architectures beyond the vanilla decoder — Mixture of Experts, Mamba and state-space models, long-context techniques (RoPE / YaRN / ring attention), diffusion language models, hybrids, encoder-only prefill (DeepSeek-V4.1-Flash’s causal encoder-decoder); plus a 5-deck companion to Raschka’s Beyond Standard LLMs (linear-attention hybrids, text diffusion, code world models, small recursive transformers, decision tree)
Training & Fine-Tuning live · 5 decks SFT pipelines, LoRA / QLoRA / DoRA / IA3, RLHF + PPO, DPO and cousins (IPO / KTO / ORPO / GRPO), Constitutional AI / RLAIF
Transformer Training live · 13 decks How a decoder-only transformer is pretrained, end to end: the loop and next-token cross-entropy, data and BPE, backpropagation through every op, SGD / momentum / Adam / AdamW, learning-rate sweeps and schedules (warmup, cosine, WSD), critical batch size, pre-LN vs post-LN, RMSNorm and µP, loss spikes (QK-norm, z-loss, clipping), mixed precision (BF16, FP16 loss scaling, FP8), sixteen bytes per parameter and activation checkpointing, data parallelism (ring all-reduce, all-gather, ZeRO/FSDP, MFU), pipeline / tensor / sequence / expert parallelism, training over slow links (top-k with error feedback, DeMo, DiLoCo, wire bytes, queueing), scaling laws (IsoFLOP, Chinchilla), checkpoints and goodput, and a bridge to fine-tuning. Every measured number comes from a 5,376-parameter decoder that really trains, labelled as a toy; a glossary links each term to its deck slide and site chapter. Related site: Training (Foundations track, after Decoder) (code), sixteen animated chapters in which the model trains in the browser
Reasoning Models live · 3 decks The o1 / R1 paradigm shift, scaling test-time compute, process vs outcome reward models, when to use reasoning models vs frontier+scaffolding
Linear Algebra for AI/ML live · 12 decks Vectors, matrices, matmul, projections (FFN up/down), eigen, SVD & LoRA, orthogonality & RoPE, gradients & backprop, attention as linear algebra, the full transformer block, tensors / einsum / FlashAttention / MoE / sharding
Mathematics for Machine Learning live · 12 decks Twelve-deck companion to Deisenroth, Faisal & Ong’s Mathematics for Machine Learning (Cambridge University Press, 2020). Part I — linear algebra, analytic geometry, matrix decompositions, vector calculus, probability, optimisation; Part II — linear regression, PCA, GMM/EM, and the kernel SVM. Lives on the Mathematics hub but re-listed here for AI-engineer discoverability

Hardware & Inference

Hardware: the machines underneath, chip to rack

The companion sites of the Hardware track, each built around animations computed by a tested model with every hardware figure sourced: GPU Kernels Explained (Hardware › Kernels: how a GPU executes the maths), Systolic Arrays Explained (Hardware › Systolic: inside a matrix engine), AI Accelerators Explained (Hardware › Accelerators: the families of AI silicon, GPUs to in-memory compute, compared on the same decoder model) and AI Racks Explained (Hardware › Racks: packages, trays, the NVL72 rack, scale-out fabrics, power, cooling, disaggregated inference on one rack, training clusters, other vendors’ designs and cost per token). The deck series below are their companions: Racks links NVIDIA GPU decks 05, 11, 13, 14, 15, 23, 24 and 25 (and InfSim 05 and 07) from its chapters, and those decks link back.

Sub-hub Status What’s inside
NVIDIA GPU Architectures live · 37 decks Pascal → Blackwell deep tour — SMs, tensor cores, memory hierarchy, NVLink, packaging, power, per-architecture low-level deep dives, the DGX Spark workstation
Google TPUs live · 12 decks Twelve-deck deep tour of the TPU programme — 2013 voice-search napkin maths through Ironwood (v7) — history, systolic arrays, every generation v1→v7, OCS & Palomar, ICI & 3D torus, XLA / JAX / Pallas software stack, TPU vs GPU
AI Accelerators live · 3 decks The families of AI silicon beyond the GPU, each run on the same decoder model: the design space and the memory wall, SRAM machines (wafer-scale, compiler-scheduled LPUs, bulk-synchronous IPUs), dataflow, meshes, edge NPUs and FPGAs, and in-memory compute; slide companion to AI Accelerators Explained
CUDA Programming live · 10 decks From your first kernel to tiled matmul, streams, atomics, Nsight profiling — visual presentations on writing CUDA from scratch

Inference

Sub-hub Status What’s inside
Local LLM Hosting live · 11 decks Self-hosting LLMs — Ollama, vLLM, llama.cpp, TGI, SGLang, Docker, multi-GPU parallelism, quantisation, determinism, production patterns
LLM Inference Simulators live · 11 decks · 1 code Building, accelerating, validating and integrating simulators for LLM inference and novel AI hardware — why architects simulate and how software-level models relate to RTL simulation, a hands-on discrete-event simulator tutorial (SimPy), prefill/decode physics and the roofline, the simulator landscape (Vidur, LLMServingSim, LLMCompass, ASTRA-sim, gem5/SST/SystemC), a live in-browser disaggregated-serving simulator, metrics / hot-spots / validation / CI, power and energy (DVFS, power caps, joules per token), simulator acceleration techniques (measured), PyTorch / ONNX Runtime / HEIR integration, a reading list, and a simulation-and-frameworks role primer
Fourier Optics for Inference live · 3 decks · 1 code Could Fourier optics accelerate disaggregated LLM inference? The physics and its bill, animated (the 2f and 4f planes, coherence, coherent detection, on-chip slab lenses and FFT meshes, a photonic link module’s signal path and bit planes, ENOB, averaging passes, conversion energy, static power), where transforms appear in inference workloads (FNet, Hyena, causality, an Amdahl analysis of prefill FLOP shares, structured weights, mask capacity), and optical prefill pools, simulated: heterogeneous pools, an optical transform engine and KV hand-off compression in the sister series’ simulator, with break-even points and a live in-browser simulator
Compute in Transit live · 5 decks · 3 code · site Doing useful work on LLM data while it crosses a link: a photonic link module (arXiv:2608.21536; a QSFP56 pluggable with an optical Fourier transform device) and its operations-per-byte budget; what LLM training moves (all-reduce, in-network aggregation, DiLoCo-family low-communication training, DCT/FFT gradient compression, DeMo) and what inference moves (KV hand-off, KV tiering, query-aware KV selection, ring attention, confidential AI); which of it fits the published operations; and the fixed-frequency-band DeMo experiment, run on GPU with the optics emulated: a fixed band, which the module could compute, ends more than a nat above DeMo’s top-k, so it is no substitute at this scale; and a SimPy training-transit simulation of that result in time and energy across data centres, in which a pluggable module never breaks even and summing the band at the switch gains only on slow WANs, where the same sum done by the GPUs gains more. Related site: Compute in Transit: Results (code), an interactive results site: the loss curves, captured energy and emulated optics replayed from the run logs, the simulation running live in the browser (its TypeScript port reproduces the recorded results.md), and the Colab notebook that ran the sweep
Simulation Engineering Toolkit live · 14 decks · 5 code The engineering around a simulator (lives under the Software and Hardware hubs; listed here for the inference-simulator thread): Rust and PyO3, SystemC TLM-2.0, DRAM/HBM modelling, cocotb co-simulation, testing frameworks, Jenkins, Jira, specifications and test plans, PyTorch and ONNX front ends for an accelerator model, performance analysis, measurement tools and methods, power, performance and area trade-offs, and an accelerator model in SimPy run end to end from a torch.export or ONNX graph
NVIDIA DeepStream live · 8 decks The streaming vision-AI SDK end-to-end — the big picture, GStreamer foundations & the NVMM/NvBufSurface memory model, the 40+ plugin toolbox, inference with TensorRT & Triton (nvinfer/nvinferserver), multi-object tracking (IOU/NvSORT/NvDCF/NvDeepSORT + Re-ID + SV3DT), the NvDsBatchMeta graph & nvdsanalytics, edge-to-cloud messaging (nvmsgconv/nvmsgbroker, Kafka/MQTT/IoT, smart record), and building & deploying (deepstream-app, pyds, Graph Composer, NGC/Kubernetes, perf tuning)

Retrieval & Multimodality

Sub-hub Status What’s inside
RAG & Retrieval Systems live · 9 decks Embedding models, vector databases, hybrid search and reranking (plus deep-dive companions on two-step cascade architecture and reranker mathematics — cross-encoder attention, ColBERT MaxSim, RankNet/LambdaRank/LambdaMART, InfoNCE, MarginMSE distillation, NDCG derivations), chunking and ingestion, agentic RAG, GraphRAG, production RAG
Vision-Language Models live · 5 decks CLIP / SigLIP, Vision Transformers, modern VLMs (Llama 3.2 V, Qwen2-VL, InternVL, Pixtral, Gemini, Claude), document AI (ColPali, Donut), and Qwen3-VL for video analytics — clip embedding, temporal-aware rerank, M-RoPE, edge-vs-cloud topology
Voice & Real-Time Agents live · 3 decks Streaming ASR + TTS, real-time frameworks (Pipecat, LiveKit Agents, OpenAI Realtime), latency budgets and turn-taking

Agents & Tools

Sub-hub Status What’s inside
Agents & Orchestration live · 8 decks · 3 code How LLM agents work, multi-agent coordination, LangGraph, framework guides (LangGraph / CrewAI / AutoGen / OpenAI Agents), a Pydantic AI deep-dive (type-safe agents, structured outputs, DI, durable execution, MCP, Logfire/evals), ReAct math agent, research-digest agent
Coding Agents Internals live · 7 decks · 1 code How Cursor, Aider, Claude Code & Codex CLI work — repo understanding, edit strategies, plus a 5-deck companion to Raschka’s Components of a Coding Agent (context, cache, tools, memory, subagents) and a runnable LangGraph implementation of the same six components
Computer-Use & Browser Agents live · 2 decks Anthropic Computer Use, OpenAI Operator, Gemini Mariner, Browser-Use, Playwright — action schemas, screen vs DOM grounding, evals
Model Context Protocol live · 5 decks Anthropic’s open protocol for connecting LLMs to tools and data — JSON-RPC primitives, transports, server-building, OAuth 2.1 security, ecosystem and patterns
Agent Harnesses Explained live · site · 10 chapters · code Interactive companion site (code): the loop, tools, context, prompt caching, permissions, sub-agents, hooks, recovery and cost of an agent harness, each chapter an animation computed by Agent_Loop_Sim, with recorded traces of a small open-weights model replayed token for token and the public coding-agent harnesses compared from their docs
Agent Protocols Explained live · site · 9 chapters · code Interactive companion site (code): MCP and A2A on the wire — JSON-RPC and the session life cycle, tools, resources and prompts, transports, sampling and elicitation, OAuth 2.1 authorisation, gateways, agent to agent and protocol security, each chapter an animation of messages sent by Agent_Loop_Sim and checked against the official MCP and A2A Python SDKs
Agent Context Explained live · site · 9 chapters · code Interactive companion site (code): context engineering, measured — BM25, dense, hybrid and reranked retrieval, chunking, packing the window, compaction, agent memory and long context against retrieval, each chapter an animation computed by Agent_Loop_Sim on a fixed SQuAD-based corpus, with a small embedding model and a cross-encoder run once offline (see also the RAG & Retrieval Systems hub)
Agent Orchestration Explained live · site · 9 chapters · code Interactive companion site (code): LangGraph’s execution model (super-steps, channels and reducers, checkpoints, interrupts, time travel), durable execution, multi-agent patterns compared on tokens, latency and success, fan-out under a rate limit, reliability and workflows at scale, each chapter an animation computed by Agent_Loop_Sim, whose graph runtime is checked against real LangGraph (see also the Agents & Orchestration hub)
Agent Evals Explained live · site · 10 chapters · code Interactive companion site (code): measuring agents — the benchmarks from their papers, pass@k and pass^k, intervals and coverage, paired comparisons, LLM judges and the swap test, trajectory grading, harness effects, contamination and cost per solved task, each chapter an animation computed by Agent_Loop_Sim’s evals module (see also the LLM Evaluations hub)
Agent Security Explained live · site · 10 chapters · code Interactive companion site (code): keeping agents safe, defences only — trust boundaries and labels, untrusted input, the lethal trifecta, least privilege and scoped tokens, sandboxing, layered detectors, plan-then-execute, approvals and fatigue, the security–utility frontier and a builder’s checklist, each chapter an animation computed by Agent_Loop_Sim’s policy module (see also the Computer-Use & Browser Agents hub)
HW Design-Space Agent live · site · code Showcase site (code) for HW_Design_Space_Agent, a LangGraph agent that explores hardware trade-offs from one spec: the LLM chooses architecture families and parameter ranges, Optuna NSGA-II searches them on a bit-exact CORDIC golden model and an Artix-7 cost model, and every run is graded against the exhaustive ground truth of 635,040 designs. Milestone 1’s eval with three real models (one run with reasoning on and off): the agent is the better selector, NSGA-II the better front-mapper; tokens and dollars per run reported. Animated trace and Pareto replays of the recorded runs

Production & Safety

Sub-hub Status What’s inside
Production LLMOps live · 4 decks LLM gateways (LiteLLM, Portkey), observability (Langfuse, Phoenix), caching strategies, cost and SLO discipline
LLM Evaluations live · 5 decks Static benchmarks vs production evals, LLM-as-judge with its biases, eval frameworks (Inspect AI, Braintrust, Phoenix, Langfuse), drift detection, red-teaming
Safety, Alignment & Red-Teaming live · 2 decks Jailbreak taxonomy, defence-in-depth (Llama Guard, ShieldGemma, NeMo Guardrails), regulatory frame (NIST AI RMF, EU AI Act)
LLM-as-a-Service (Cloud *aaS 06) live · 1 deck Managed LLM service providers — pricing & latency comparison (OpenAI / Anthropic / Bedrock / Vertex / Azure OpenAI / Together / Groq / Fireworks / Replicate), prompt caching, embeddings & RAG-as-a-Service, agents-as-a-Service, fine-tuning APIs, evals / observability, MCP server hosting, LLM-specific security and governance. Companion to Local LLM Hosting (the self-hosted side) and part of the Cloud *aaS series on the Software index

LLM History

Sub-hub Status What’s inside
LLM History live · 10 decks A history of large language models from Shannon (1948) to the multipolar 2026 frontier — people, labs and ideas. Pre-Transformer NLP, the 2017 transformer paper and its eight authors, university labs, OpenAI, Anthropic, Google DeepMind, Meta/Mistral/xAI/Cohere, the Chinese frontier labs (DeepSeek, Qwen, Kimi, GLM, Yi), and a forecast deck on future directions. Equal attention to technical detail and the human story behind the field

Anthropic Claude

Sub-hub Status What’s inside
Anthropic Claude live · 1 deck Comprehensive presentation series on Claude — models, Claude.ai, Claude Code, API and SDKs, integrations, agents, productivity, Claude Desktop, hands-on workflow walkthroughs

Key Publications

Sub-hub Status What’s inside
Key LLM Publications live · 10 decks Fifty essential papers, articles and blog posts every practising LLM/agent engineer should know — ten themed decks of five, each publication with a plain-language summary, an engineer’s interpretation, and diagrams of the core idea. Foundations (Transformer, BERT, GPT-3, scaling laws, Chinchilla); alignment & fine-tuning (InstructGPT/RLHF, Constitutional AI, LoRA/QLoRA, DPO); reasoning & test-time compute (CoT, self-consistency, Tree of Thoughts, process rewards, DeepSeek-R1); efficient inference (FlashAttention, MoE, GPTQ, PagedAttention/vLLM, speculative decoding); context, position & retrieval (RoPE, RAG, DPR, lost-in-the-middle, Self-RAG); agents & tool use (ReAct, Toolformer, Reflexion, Building Effective Agents, MCP); evaluation & benchmarks (MMLU, HumanEval, BIG-bench, HELM, LLM-as-Judge); multimodal models (ViT, CLIP, Flamingo, LLaVA, Whisper); open models & architectures (LLaMA, Llama 2, Mistral, Mixtral, DeepSeek-V3); safety, jailbreaks & security (GCG, prompt injection, Llama Guard, Sleeper Agents, weak-to-strong)

Interpretability

Sub-hub Status What’s inside
The Transformer Circuits Thread live · 6 decks Anthropic’s mechanistic-interpretability research from transformer-circuits.pub, presented for engineers as six single-publication deep dives tracing one arc: A Mathematical Framework for Transformer Circuits (residual stream, QK/OV circuits, composition); In-context Learning & Induction Heads (the phase change behind in-context learning); Toy Models of Superposition (why neurons are polysemantic); Towards Monosemanticity (sparse autoencoders / dictionary learning recover interpretable features); Scaling Monosemanticity (features extracted from Claude 3 Sonnet, feature steering, “Golden Gate Claude”); and On the Biology of a Large Language Model (circuit tracing with attribution graphs on Claude 3.5 Haiku). Complements the broad Key LLM Publications canon and the Safety, Alignment & Red-Teaming hub

Articles & Reference

Project Description
LLM Articles Articles and presentations on batching, KV cache optimisation, and quantisation for inference

How this is organised