Further Learning

Topics We'll Cover

• A Four-Week Learning Path
• Books: Simulation and Performance Analysis
• Books: Computer and Accelerator Architecture
• Courses and Lecture Series
• Papers: LLM Serving and Disaggregation
• Papers: Simulators and Simulation Methodology
• Hands-On Tutorials and Documentation
• Source Code Worth Reading
• FHE and Photonic Computing
• Power and Energy
• Related Decks on This GitHub

A Four-Week Learning Path

One evening a day for four weeks, assuming solid Python and computer architecture already. Each week ends with something built.

Books: Simulation and Performance Analysis

• Averill M. Law, Simulation Modeling and Analysis (McGraw-Hill, 5th ed., 2015)The standard text on discrete-event simulation: input modelling, random-variate generation, output analysis, warm-up, replications, variance reduction. The statistics chapters underpin deck 06.
• Jerry Banks, John S. Carson II, Barry L. Nelson & David M. Nicol, Discrete-Event System Simulation (Pearson, 5th ed., 2010)A gentler companion to Law, with clear treatment of event scheduling and process interaction, and verification and validation.
• Mor Harchol-Balter, Performance Modeling and Design of Computer Systems: Queueing Theory in Action (Cambridge University Press, 2013)Queueing theory written for systems engineers. Little's law, M/G/1, processor sharing, scheduling: exactly the analytic checks a simulator should pass.
• Edward D. Lazowska, John Zahorjan, G. Scott Graham & Kenneth C. Sevcik, Quantitative System Performance (1984), free onlineThe operational laws (utilisation, Little, forced flow) and mean-value analysis. Old, short, and still the best source for back-of-envelope performance reasoning.
• Raj Jain, The Art of Computer Systems Performance Analysis (Wiley, 1991)Experimental design, measurement, simulation and common mistakes in performance studies. The chapters on presenting results are as useful as the maths.
• Richard M. Fujimoto, Parallel and Distributed Simulation Systems (Wiley, 2000)The reference on PDES: conservative and optimistic synchronisation, Time Warp, lookahead (deck 08).
• Brendan Gregg, Systems Performance (Addison-Wesley, 2nd ed., 2020)Profiling methodology and tools: the "profile first" habit applied to real systems and to your own simulator.

Books: Computer and Accelerator Architecture

• John L. Hennessy & David A. Patterson, Computer Architecture: A Quantitative Approach (Morgan Kaufmann, 6th ed., 2017)The quantitative method itself. Chapter 2 and appendix B cover the memory hierarchy (caches, virtual memory, DRAM organisation); chapter 7 covers domain-specific architectures including the TPU.
• Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang & Joel Emer, Efficient Processing of Deep Neural Networks (Morgan & Claypool, 2020)Dataflows, memory hierarchies and evaluation methodology for DNN accelerators; the conceptual basis of Timeloop and Accelergy.
• Lieven Eeckhout, Computer Architecture Performance Evaluation Methods (Morgan & Claypool Synthesis Lectures, 2010)A compact survey of exactly this series' subject for CPUs: analytical models, simulation types, sampling (SimPoint, SMARTS), statistical simulation.
• Bruce Jacob, Spencer Ng & David Wang, Memory Systems: Cache, DRAM, Disk (Morgan Kaufmann, 2007)How memory really behaves (banks, refresh, scheduling), for when "bandwidth × efficiency" stops being good enough.
• Natalie Enright Jerger, Tushar Krishna & Li-Shiuan Peh, On-Chip Networks (Morgan & Claypool, 2nd ed., 2017)Topologies, routing, flow control and NoC modelling (the authors are behind the Garnet network model in gem5).
• David C. Black & Jack Donovan, SystemC: From the Ground Up (Springer, 2nd ed., 2010)The practical introduction to SystemC and TLM-2.0 virtual-platform modelling. Pair it with the IEEE 1666 standard.

Courses and Lecture Series

• MIT 6.5940 — TinyML and Efficient Deep Learning Computing (Song Han)Quantisation, pruning, efficient attention, LLM serving systems. Lectures and labs free online.
• MIT 6.5930/1 — Hardware Architecture for Deep Learning (Sze & Emer)Accelerator design and evaluation; labs use Timeloop and Accelergy.
• CMU 10-414/714 — Deep Learning Systems (Tianqi Chen & Zico Kolter)Build a framework from scratch: autodiff, kernels, hardware acceleration. Excellent background for framework integration.
• Stanford CS149 — Parallel ComputingParallel programming models, GPUs, and the performance reasoning behind accelerators.
• Onur Mutlu's Computer Architecture lectures (ETH Zürich and CMU, on YouTube)Hundreds of hours on memory systems, DRAM, processing-in-memory and simulation infrastructure; dense and current.
• How to Scale Your Model (Google DeepMind, free online book)Rooflines, sharding and inference arithmetic for transformers on TPUs and GPUs. The best single source for the numbers in deck 03.
• Conference tutorials (ISCA, MICRO, ASPLOS, HPCA)gem5, ASTRA-sim, SST and Timeloop teams run hands-on tutorials whose slides and recordings are usually posted on the projects' sites.

Papers: LLM Serving and Disaggregation

• Williams, Waterman & Patterson, "Roofline: An Insightful Visual Performance Model for Multicore Architectures" (CACM, 2009)The model behind every cost model in this series.
• Pope et al., "Efficiently Scaling Transformer Inference" (MLSys 2023)Analytical partitioning of inference across chips; how to reason about latency against cost.
• Yu et al., "Orca: A Distributed Serving System for Transformer-Based Generative Models" (OSDI 2022)Iteration-level scheduling: continuous batching.
• Kwon et al., "Efficient Memory Management for Large Language Model Serving with PagedAttention" (SOSP 2023)vLLM and paged KV.
• Agrawal et al., "Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve" (OSDI 2024)Chunked prefill and stall-free batching.
• Patel et al., "Splitwise: Efficient Generative LLM Inference Using Phase Splitting" (ISCA 2024)Production characterisation and the case for prompt and token pools.
• Zhong et al., "DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving" (OSDI 2024)Goodput, and simulation-driven placement.
• Qin et al., "Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving" (FAST 2025)Disaggregation at production scale with a distributed KV store.
• Yuan et al., "LLM Inference Unveiled: Survey and Roofline Model Insights" (2024)A survey of inference optimisations through the roofline lens; introduces LLM-Viewer.

Papers: Simulators and Simulation Methodology

• Agrawal et al., "Vidur: A Large-Scale Simulation Framework for LLM Inference" (MLSys 2024)The reference serving simulator: operator profiling, predictors, scheduler models, configuration search.
• Cho et al., "LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale" (IISWC 2024)Serving simulation coupled to accelerator and network simulators.
• Zhang et al., "A Hardware Evaluation Framework for Large Language Model Inference" (LLMCompass, ISCA 2024)Block-level accelerator description with mapping, area and cost.
• Won et al., "ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale" (ISPASS 2023)Layered workload, system and network simulation; Chakra traces.
• Sherwood, Perelman, Hamerly & Calder, "Automatically Characterizing Large Scale Program Behavior" (SimPoint, ASPLOS 2002)Representative-interval sampling.
• Wunderlich, Wenisch, Falsafi & Hoe, "SMARTS: Accelerating Microarchitecture Simulation via Rigorous Statistical Sampling" (ISCA 2003)Statistical sampling with confidence bounds.
• Chandy & Misra, "Distributed Simulation: A Case Study in Design and Verification of Distributed Programs" (IEEE TSE, 1979) and Jefferson, "Virtual Time" (ACM TOPLAS, 1985)The founding papers of conservative and optimistic PDES.

Hands-On Tutorials and Documentation

• SimPy documentationThe "SimPy in 10 minutes" guide, topical guides on processes and shared resources, and worked examples (bank, car wash, machine shop).
• PyTorch: Custom backends for torch.compile and Facilitating new backend integration with PrivateUse1The two official routes for putting a simulator behind PyTorch (deck 09).
• ONNX Runtime: Add a new execution providerThe EP interface, partitioning and registration.
• MLIR Toy tutorialBuild a small language on MLIR: dialects, lowering, interfaces. The fastest way into MLIR, and so into HEIR.
• HEIR documentationPipelines, dialects, backends and getting-started guides for the FHE compiler.
• Perfetto documentation and ui.perfetto.devTrace formats, SQL trace analysis and the UI.
• Hypothesis, cocotb and Verilator (its coverage explained)Property-based testing, Python testbenches for RTL, and the fast open-source RTL simulator.
• PyO3 user guideWriting Python extensions in Rust: the route to a compiled simulator core with a Python front end.

Source Code Worth Reading

Reading a mature simulator's source teaches more about structure than any book. Start with the smaller ones.

FHE and Photonic Computing

Fully homomorphic encryption

• Craig Gentry, "A Fully Homomorphic Encryption Scheme" (PhD thesis, Stanford, 2009)Where it started: bootstrapping.
• Cheon, Kim, Kim & Song, "Homomorphic Encryption for Arithmetic of Approximate Numbers" (CKKS, ASIACRYPT 2017)The scheme of choice for encrypted machine learning.
• Samardzic et al., "F1: A Fast and Programmable Accelerator for Fully Homomorphic Encryption" (MICRO 2021), and the follow-up CraterLake (ISCA 2022)The landmark FHE accelerator papers: why NTT throughput and on-chip memory bandwidth dominate. See also BTS (ISCA 2022) and ARK (MICRO 2022).
• FHE.org resourcesA community-maintained index of tutorials, talks, libraries and papers.

Photonic and optical computing

• Joseph W. Goodman, Introduction to Fourier Optics (W. H. Freeman, 4th ed., 2017)Why a lens computes a Fourier transform; the physics behind optical transform engines.
• Shen et al., "Deep Learning with Coherent Nanophotonic Circuits" (Nature Photonics, 2017)Matrix multiplication in Mach–Zehnder interferometer meshes.
• Wetzstein et al., "Inference in Artificial Intelligence with Deep Optics and Photonics" (Nature, 2020) and Shastri et al., "Photonics for Artificial Intelligence and Neuromorphic Computing" (Nature Photonics, 2021)Reviews of the field: opportunities, and the conversion, precision and memory challenges a system simulator must capture.
• Optical-computing companies' own published articlesSeveral companies write publicly about optical computing for AI and FHE acceleration; read the material of any team whose work you want to understand.

Related Decks on This GitHub

• Inside vLLM: PagedAttention and Continuous BatchingThe serving engine whose scheduler the simulator models.
• Multi-GPU ParallelismTP, PP and EP in practice: the communication terms deck 03 left out.
• Key Publications: Efficient InferenceFlashAttention, MoE, GPTQ, PagedAttention and speculative decoding, summarised.
• AI_MMUL_Unit and LLM_Transformer_Decoder_RTLRTL implementations with test suites: the other end of the fidelity ladder from deck 01.
• Interview_RTL_LLM_AcceleratorsLLM-accelerator design and verification challenges, including memory, dataflow and system integration.
• FHE Accelerator Simulators (sister series)The same methods applied to an FHE accelerator running CKKS bootstrapping, with OpenFHE traces, HEIR-compiled programs and optical NTT engines; its glossary covers FHE in depth.
• Cryptography series (Mathematics hub)The mathematics behind FHE (deck 08) and crypto hardware accelerators, including NTT hardware (deck 10).
• NVIDIA GPU Architectures and Google TPUsThe incumbents whose datasheets fill the cost models.

Power and Energy

• Mark Horowitz, "Computing's Energy Problem (and What We Can Do About It)" (ISSCC 2014 keynote paper)The energy-per-operation table everyone quotes: why data movement, not arithmetic, sets the energy budget.
• Luiz André Barroso & Urs Hölzle, "The Case for Energy-Proportional Computing" (IEEE Computer, 2007), and Barroso, Hölzle & Ranganathan, The Datacenter as a Computer (Morgan & Claypool, 3rd ed., 2018)Static power, utilisation and energy proportionality; PUE and datacentre power provisioning.
• Li et al., "McPAT: An Integrated Power, Area, and Timing Modeling Framework for Multicore and Manycore Architectures" (MICRO 2009), with source, and the CACTI SRAM and cache modelArchitecture-level power from structure; the standard companions to gem5.
• Accelergy (Wu, Emer & Sze, ICCAD 2019)Energy estimation for accelerator designs from per-action energy tables, paired with Timeloop for mappings.
• Patel et al., "POLCA: Power Oversubscription in LLM Cloud Providers" (2023)How much power LLM inference really draws, and how to oversubscribe and cap safely.
• Stojkovic et al., "DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency" (2024)Reconfiguring instances, parallelism and GPU frequency with load to save energy within SLOs.
• ML.ENERGY leaderboard and Zeus (Zeus explained)Measured energy per request for open models on real GPUs, and the library to measure your own: baselines for validating an energy model.
• NVIDIA DCGM documentation (DCGM explained)Reading power, clocks and throttle reasons from GPUs: the data you regress pJ/FLOP and pJ/byte from (deck 07).