Simulator Development — a Hands-On Tutorial

Topics We'll Cover

• Anatomy of a Discrete-Event Simulator
• Step 1 — A Kernel in 30 Lines
• Step 2 — A Queue, Checked Against Theory
• Interactive: Step Through the Event List
• Step 3 — Processes: Generators as Coroutines
• SimPy: The Whole Vocabulary on One Page
• Modelling Hardware in SimPy
• Shared Bandwidth: Hold the Link or Share It?
• Structuring a Simulator That Lasts
• Time, Ordering and Determinism Pitfalls
• Making the Simulator Fast
• Beyond Python: C++, Rust, SystemC
• Exercises
• What to Take Away

Anatomy of a Discrete-Event Simulator

A discrete-event simulator (DES) models a system whose state only changes at instants: a request arrives, a transfer finishes, a batch completes. Between those instants nothing interesting happens, so the simulator jumps straight from one event to the next. This is next-event time advance, and it is why a DES can cover an hour of traffic in seconds.

Step 1 — A Kernel in 30 Lines

Write this once yourself before using a library. Every DES library, SimPy included, is a refinement of it.

import heapq, itertools

class Sim:
    def __init__(self):
        self.now = 0.0
        self._q = []                       # heap of (time, seq, handler, args)
        self._seq = itertools.count()      # tie-breaker: FIFO among equal times

    def schedule(self, delay, handler, *args):
        assert delay >= 0, "no time travel"
        heapq.heappush(self._q, (self.now + delay, next(self._seq), handler, args))

    def run(self, until=float("inf")):
        while self._q and self._q[0][0] <= until:
            t, _, handler, args = heapq.heappop(self._q)
            self.now = t
            handler(*args)
        self.now = min(until, self.now) if self._q else self.now

• The sequence number matters. Without it, two events at the same time are ordered by comparing handlers, which is an error in Python 3 and nondeterministic in other languages.
• A binary heap gives O(log n) insert and pop. Calendar queues and ladder queues can reach O(1) for large event lists, but the heap is almost always good enough.
• No cancellation yet. The usual trick is a "cancelled" flag that the loop skips (lazy deletion) rather than removing the entry from the heap.

Step 2 — A Queue, Checked Against Theory

The first model to build on any new kernel is the single-server queue, because queueing theory gives you its exact answer. With Poisson arrivals (rate λ) and exponential service (rate μ), the M/M/1 mean time in system is W = 1 / (μ − λ).

import random
sim, rng = Sim(), random.Random(42)
LAM, MU = 0.8, 1.0
queue, busy, times = [], [False], []

def arrive():
    queue.append(sim.now)
    sim.schedule(rng.expovariate(LAM), arrive)
    if not busy[0]: start()

def start():
    busy[0] = True
    sim.schedule(rng.expovariate(MU), depart, queue.pop(0))

def depart(t_arrived):
    times.append(sim.now - t_arrived)
    busy[0] = False
    if queue: start()

sim.schedule(0, arrive); sim.run(until=200_000)
print(sum(times) / len(times), "vs theory", 1 / (MU - LAM))   # ~5.0 vs 5.0

Interactive: Step Through the Event List

A two-stage pipeline: jobs arrive, are computed on one unit, then transferred over one link. Step through and watch the clock jump between event times, the event list re-order, and queues form when the link is slower than the compute.

The kernel underneath is exactly the 30-line one from slide 02: a sorted list of (time, sequence, kind, job). Make the transfer slower than the compute and the link queue grows without bound: the link is the hot-spot.

Step 3 — Processes: Generators as Coroutines

Event-scheduling code scatters one job's life across several handlers (arrive, start, depart). The process-interaction worldview writes each entity's life as one sequential function that suspends whenever it has to wait. Python generators make this natural: yield hands an event to the kernel, and the kernel resumes the generator when that event fires.

import simpy, random
rng = random.Random(42)
env = simpy.Environment()
server = simpy.Resource(env, capacity=1)
times = []

def job(env):
    t0 = env.now
    with server.request() as req:
        yield req                     # wait for the server
        yield env.timeout(rng.expovariate(1.0))
    times.append(env.now - t0)

def source(env):
    while True:
        yield env.timeout(rng.expovariate(0.8))
        env.process(job(env))

env.process(source(env))
env.run(until=200_000)

SimPy: The Whole Vocabulary on One Page

SimPy (v4, pure Python, MIT-licensed) is small enough to learn in an afternoon. This table is most of it.

Modelling Hardware in SimPy

Most SoC building blocks map onto three primitives. A bounded FIFO between a producer and a consumer shows the most important one: back-pressure.

fifo = simpy.Store(env, capacity=4)            # put() blocks when full: back-pressure
hbm  = simpy.Resource(env, capacity=2)         # two memory channels

def dma(env, tiles):
    for tile in tiles:
        with hbm.request() as ch:
            yield ch
            yield env.timeout(LAT + tile.nbytes / CH_BW)   # cost model, not magic numbers
        yield fifo.put(tile)                         # stalls here if compute falls behind

def compute(env, stats):
    while True:
        tile = yield fifo.get()                      # stalls here if DMA falls behind
        t0 = env.now
        yield env.timeout(tile.flops / PEAK)
        stats.busy += env.now - t0                   # utilisation probe

Shared Bandwidth: Hold the Link or Share It?

The most common modelling decision in a data-movement simulator, and the one most often made by accident. Two transfers want the same link at once. What happens?

Same throughput, same utilisation, different latency distribution. If your metric is a tail latency, this choice can change the answer. Write it down in the model's specification and test it.

Structuring a Simulator That Lasts

Simulators that architects actually use share one structure: separate the things that change at different rates.

• The cost model is a plug-in. The engine asks "how long does this batch take?" and never knows whether the answer came from a formula, a lookup table measured on RTL, or a regression. This is how the model improves over the project without rewrites. (Vidur does exactly this with profiled operator tables.)
• Configuration is data, not code. Every run should be reproducible from one config file plus a seed. Stamp outputs with the config hash and the git SHA.
• Probes are passive. Measuring must never change behaviour. Keep metric code out of scheduling decisions so you can switch tracing off for speed.
• One clock unit everywhere. Choose seconds (floats) or integer picoseconds and convert at the edges.

Time, Ordering and Determinism Pitfalls

Making the Simulator Fast

Simulator speed is an architecture-team productivity metric: a sweep of 500 configurations at 10 minutes each is a three-day wait. In order of payoff (deck 08 takes each of these further, with measurements):

1. Measure first. cProfile or py-spy for where the time goes; count events per simulated second. Most slow simulators are slow because of probes, logging or per-event object allocation, not the kernel.
2. Simulate fewer events. Model a whole batch step as one event, not one per token or per layer. Replace a detailed sub-model with an analytic one where it does not affect the answer (a fluid approximation for bulk transfers, say).
3. Run sweeps in parallel. Independent configurations are embarrassingly parallel: multiprocessing, a job array, or Jenkins agents. This usually beats any single-run optimisation.
4. Try a faster interpreter such as PyPy, which suits generator-heavy SimPy code; measure, as gains vary.
5. Move the hot path to a compiled language. Keep the Python API and configuration, and push the event loop and cost models into C++ or Rust (pybind11, PyO3/maturin).
6. Parallel discrete-event simulation (PDES) is the last resort: conservative (Chandy–Misra–Bryant, needs lookahead) or optimistic (Time Warp, needs rollback). Powerful, but it imposes structure on the whole model; SST and parallel SystemC kernels go this way.

Beyond Python: C++, Rust, SystemC

use std::{cmp::Reverse, collections::BinaryHeap};

#[derive(PartialEq, Eq, PartialOrd, Ord)]
struct Ev { t_ps: u64, seq: u64, kind: Kind }   // integer picoseconds: exact ordering

fn run(q: &mut BinaryHeap<Reverse<Ev>>, model: &mut Model) {
    while let Some(Reverse(ev)) = q.pop() {
        model.now = ev.t_ps;
        model.handle(ev.kind, q);          // handlers push follow-on events
    }
}

Exercises

Each takes an evening and builds a skill this series relies on. The Disaggregated_Inference_Sim repo is the worked solution for the last three.

1. Type in the 30-line kernel. Add event cancellation with lazy deletion. Reproduce the M/M/1 result and plot simulated against theoretical W for ρ from 0.1 to 0.95. Watch the variance grow near saturation.
2. Rewrite the M/M/1 in SimPy and time both versions. How many events per second does each manage?
3. Model the DMA → FIFO → compute pipeline of slide 07. Sweep FIFO depth from 1 to 16 and find where throughput stops improving.
4. Implement the link both ways (FCFS hold and processor sharing). Show that the mean throughput matches and the latency distributions differ.
5. Add a probe layer: per-resource utilisation, time-weighted queue length, and a Chrome-trace exporter. Open the trace in Perfetto.
6. Put a roofline cost model behind an interface, then replace it with a lookup table without touching the engine.
7. Write a test that checks Little's law (L = λW) on your model, and wire it into CI.
8. Attach an energy model: static watts × time plus energy per operation and per byte moved. Report joules per job, and add a power cap that stretches compute-bound work (deck 07).

What to Take Away

• A DES is a priority queue and a loop. Write one by hand once; then every library is familiar.
• Validate the kernel against queueing theory before trusting it with anything that has no closed form.
• SimPy's process-interaction style (generators, Resource, Store, Container) maps naturally onto engines, links, FIFOs and capacity.
• Separate workload, config, cost model, engine and probes. The cost model will be replaced several times in a project; make that cheap.
• Determinism, integer time and per-component RNG streams save weeks of debugging.
• Speed comes from fewer events and parallel sweeps first, compiled kernels second, PDES last.