Rust for Simulation Engineers

Topics We'll Cover

• Why Rust for Simulators
• The Event List: BinaryHeap and Reverse
• Ordering Floats, Breaking Ties
• Interactive: The Event Queue's Ordering Rule
• Events as Enums, Models as Traits
• Ownership in an Event Loop
• From SimPy Processes to State Machines
• Traits for Pluggable Cost Models
• Errors: Result, the ? Operator, and When to Panic
• Fearless Parallel Sweeps
• The Toolchain: cargo, clippy, rustfmt, criterion
• Testing in Rust: cargo test and proptest
• What to Take Away

Why Rust for Simulators

A discrete-event simulator is a long-running loop over a priority queue that mutates a lot of shared state. Python (SimPy) is the right place to start: quick to write, easy to read, a good reference model. When the same model has to run thousands of configurations a night, the core is usually rewritten in a compiled language. LLM Inference Simulators 02, slide 12 sketched the options; this deck does the Rust one properly, using the code in Rust_DES_Kernel.

The Event List: BinaryHeap and Reverse

Every discrete-event kernel is a priority queue of future events and a clock. Rust's standard library has the queue: std::collections::BinaryHeap. It is a max-heap, so wrapping each entry in std::cmp::Reverse turns it into the min-heap a simulator needs: the earliest event comes out first.

/// The event queue and the clock.
#[derive(Debug)]
pub struct Scheduler<E> {
    now: Time,
    seq: u64,
    processed: u64,
    // `BinaryHeap` is a max-heap; `Reverse` turns it into the min-heap a DES needs.
    heap: BinaryHeap<Reverse<Entry<E>>>,
}

/// Schedule `event` at absolute time `at` (never in the past).
pub fn schedule_at(&mut self, at: Time, priority: Priority, event: E) {
    assert!(!at.is_nan(), "event time is NaN");
    assert!(
        at >= self.now,
        "event scheduled in the past: {at} < {}",
        self.now
    );
    self.heap.push(Reverse(Entry {
        time: at,
        priority,
        seq: self.seq,
        event,
    }));
    self.seq += 1;
}

/// Remove the next event and advance the clock to its time.
pub fn pop(&mut self) -> Option<(Time, E)> {
    let Reverse(e) = self.heap.pop()?;
    self.now = e.time;
    self.processed += 1;
    Some((e.time, e.event))
}

• Push and pop are O(log n). Calendar and ladder queues can do better for huge event lists, but a binary heap is almost always enough (the criterion numbers on slide 11 are with this heap).
• The clock only moves forward. The assert! turns "an event in the past" from a silent wrong answer into an immediate, located failure.
• schedule_in computes now + delay once, as SimPy does. That detail matters when you want two languages to agree to the last bit (deck 02).

Ordering Floats, Breaking Ties

BinaryHeap needs its entries to be Ord: a total order. f64 is only PartialOrd, because NaN compares unequal to everything, itself included. Rust makes you choose what to do about that, where C++ and Python let it slide.

// `f64` is not `Ord` (NaN), so order by `total_cmp`. Times are never NaN:
// `schedule` rejects them.
impl<E> Ord for Entry<E> {
    fn cmp(&self, other: &Self) -> Ordering {
        self.time
            .total_cmp(&other.time)
            .then(self.priority.cmp(&other.priority))
            .then(self.seq.cmp(&other.seq))
    }
}

Interactive: The Event Queue's Ordering Rule

Schedule events, then pop them. The queue below is a real binary heap (the same algorithm as BinaryHeap). Switch the tie-break rule to see why the sequence number is there: without it, the order of simultaneous events depends on the order the heap happens to hold them in.

The "unit-test example" is the kernel's own test earlier_first_then_urgent_then_fifo: popping should give urgent, n1, n2, late. With six simultaneous events and no sequence number, the pop order is not the insertion order, and it changes if you insert in a different order: harmless here, fatal for reproducibility.

Events as Enums, Models as Traits

In Python an event is usually a callback. In Rust it is more natural to make events plain data, an enum, and to let the model decide what each one means. The compiler then checks that every handler deals with every kind of event.

/// A model reacts to one event at a time and may schedule more.
pub trait Model {
    type Event;

    /// Handle one event. Return `false` to stop the run.
    fn handle(&mut self, sched: &mut Scheduler<Self::Event>, event: Self::Event) -> bool;
}

/// Run `model` until it asks to stop or the queue empties. Returns the final time.
pub fn run<M: Model>(model: &mut M, sched: &mut Scheduler<M::Event>) -> Time {
    while let Some((_, ev)) = sched.pop() {
        if !model.handle(sched, ev) {
            break;
        }
    }
    sched.now()
}

impl Model for Md1 {
    type Event = Event;

    fn handle(&mut self, s: &mut Scheduler<Event>, ev: Event) -> bool {
        match ev {
            Event::Arrive => {
                self.arrived += 1;
                if self.arrived < self.customers {
                    let gap = self.rng.expovariate(self.rate);
                    s.schedule_in(gap, Event::Arrive);
                }
                if self.busy {
                    self.queue.push_back(s.now());
                } else {
                    self.start_service(s, s.now());
                }
            }
            Event::Depart => {
                self.served += 1;
                self.busy = false;
                if let Some(t) = self.queue.pop_front() {
                    self.start_service(s, t);
                }
            }
        }
        self.served < self.customers
    }
}

• Exhaustive match. Add an event variant and every match that ignores it stops compiling. A forgotten callback in Python is a silent no-op.
• Events are small and Copy: a tag and an index. An Ev is 16 bytes and a heap entry, with its time, priority and sequence number, 40 (pinned by a test), rather than a closure with captured state.
• This M/D/1 model is checked against the Pollaczek–Khinchine formula in tests/kernel.rs (within 3% at loads 0.3, 0.6 and 0.8). An analytic check is the first test any new kernel should pass.

Ownership in an Event Loop

The first design most people write holds a reference to a request while calling a method on the simulation that owns it. The borrow checker rejects it, and it is right to: finish() could push to requests, reallocate the vector and leave r dangling. This is the real compiler output:

impl Sim {
    fn finish(&mut self, r: &mut Request) {
        r.finish = Some(self.now);
        self.done += 1;
    }

    fn step(&mut self) {
        let r = &mut self.requests[0];   // a mutable borrow of part of self ...
        self.finish(r);                  // ... while self is borrowed mutably again
    }
}

impl Sim {
    fn finish(&mut self, rid: usize) {
        self.requests[rid].finish = Some(self.now);
        self.done += 1;
    }

    fn step(&mut self) {
        let rid = 0;          // events and queues carry indices into an arena
        self.finish(rid);
    }
}

error[E0499]: cannot borrow `*self` as mutable more than once at a time
  --> src/main.rs:15:9
   |
14 |         let r = &mut self.requests[0];   // a mutable borrow of part of self ...
   |                      ------------- first mutable borrow occurs here
15 |         self.finish(r);                  // ... while self is borrowed mutably again
   |         ^^^^        - first borrow later used here
   |         |
   |         second mutable borrow occurs here

The pattern is an arena with indices: the simulation owns every request and instance in a Vec; events, queues and batches carry usize indices; each borrow lasts one statement. Where a loop must call &mut self methods while walking a collection, take the collection out and put it back:

// Take the batch out so the loop can call `finish(&mut self)`.
let running = std::mem::take(&mut self.inst[i].running);
let mut still = Vec::with_capacity(running.len());
for r in running {
    let q = &mut self.reqs[r];
    q.itls.push(now - q.last_token);
    q.tokens_out += 1;
    q.last_token = now;
    if q.tokens_out >= q.output_len {
        self.inst[i].kv_used -= self.kv_need(r);
        self.finish(s, r);
    } else {
        still.push(r);
    }
}
self.inst[i].running = still;

From SimPy Processes to State Machines

In SimPy, a prefill instance is a generator: "form a batch, yield env.timeout(step), emit tokens, loop". Each yield is a point where the process sleeps. Without stable generators, the Rust engine names those sleep points as states and makes each event resume the instance from where it slept.

fn submit(&mut self, s: &mut Scheduler<Ev>, i: usize, rid: usize) {
    let inst = &mut self.inst[i];
    inst.queue.push_back(rid);
    if inst.waiting && !inst.wake_triggered {
        inst.wake_triggered = true;
        s.schedule_now(Ev::Wake(i));
    }
}

The flag wake_triggered mirrors SimPy's rule that an event already triggered is not triggered again. It matters only when two requests reach a waiting instance before its wake-up runs, and ordinary Poisson workloads never do that. A bursty workload does: requests queue behind a busy prefill, leave in one batch, and their KV transfers finish together into an idle decoder. That case is now a golden test; with the flag removed, the engine wakes the decoder twice and its invariant check stops the run with "a step was in flight" (slide 09). Async Rust is the other route to process-style code; the explicit machine was chosen here because its event order is easy to reason about and to match against SimPy.

Traits for Pluggable Cost Models

The most important structural decision in an architecture simulator is to keep time in one replaceable place (LLM Inference Simulators 02, slide 09). In Rust that place is a trait. This example compiles and its tests pass (snippets/RESULTS.md):

/// What every cost model must answer. The engine knows nothing else about time.
pub trait CostModel {
    fn prefill(&self, tokens: u64) -> f64;
    fn decode(&self, batch: u64, context: u64) -> f64;
}

/// Static dispatch: one copy of the engine per cost model, every call inlinable.
pub fn total_decode_time<C: CostModel>(cost: &C, steps: u64) -> f64 {
    (0..steps).map(|k| cost.decode(8, 4096 + 8 * k)).sum()
}

Rust_DES_Kernel itself uses one concrete CostModel struct, because its job is to mirror one Python class exactly. A trait is the next step when a second model (a measured table, a calibrated fit) has to plug in.

Errors: Result, the ? Operator, and When to Panic

Rust separates two kinds of failure, and a simulator has both.

pub fn build(&self) -> Result<SimConfig, ConfigError> {
    let unknown = |what: &str, v: &str| ConfigError(format!("unknown {what} {v:?}"));
    let names = [
        Some(&self.model),
        Some(&self.device),
        self.prefill_device.as_ref(),
        self.decode_device.as_ref(),
    ];
    for key in names.into_iter().flatten() {
        if let Some(why) = hardware::python_js_only(key) {
            return Err(ConfigError(format!("{key} {why}")));
        }
    }
    if let Some(t) = &self.kv_transit {
        return Err(ConfigError(format!(
            "kv_transit {t:?}: KV hand-off compression is Python and JS only, not in the Rust port"
        )));
    }
    let dev = |key: &Option<String>| -> Result<Option<Accelerator>, ConfigError> {
        key.as_ref()
            .map(|k| hardware::device(k).ok_or_else(|| unknown("device", k)))
            .transpose()
    };
    let mode = match self.mode.as_str() {
        "disagg" => Mode::Disagg,
        "colocated" => Mode::Colocated,
        m => return Err(unknown("mode", m)),
    };
    let mut link = hardware::link(&self.link).ok_or_else(|| unknown("link", &self.link))?;

/// Run one simulation. Returns (per-request stamps, summary JSON, events processed,
/// seconds simulating, seconds summarising).
#[pyfunction]
#[pyo3(signature = (config_json, rows, with_summary=true))]
fn simulate_json(
    py: Python<'_>,
    config_json: &str,
    rows: Vec<Row>,
    with_summary: bool,
) -> PyResult<Output> {
    let spec = spec(config_json)?;
    // The GIL is released for the whole run: nothing in here touches Python.
    py.detach(|| run_one(&spec, &rows, with_summary))
        .map_err(PyValueError::new_err)
}

Fearless Parallel Sweeps

Architecture questions are answered by sweeps: rates, pool sizes, links, power caps. Independent runs share nothing, so they parallelise perfectly. The rayon crate turns iter() into par_iter(), and the compiler checks the rest.

// Independent runs share nothing, so rayon can farm them out safely:
// the compiler checks that each closure only reads `cfg`.
let rows: Vec<String> = a
    .sweep
    .par_iter()
    .map(|&rate| {
        let m = summarise(&simulate(cfg.clone(), wl(rate)).expect("valid config"));
        let g = |p: &[&str]| {
            p.iter()
                .fold(&m, |v, k| &v[*k])
                .as_f64()
                .unwrap_or(f64::NAN)
        };
        format!(
            "{rate},{:.6},{:.6},{:.4},{:.4}",
            g(&["latency_s", "ttft", "p99"]),
            g(&["latency_s", "tpot", "p99"]),
            g(&["throughput", "slo_attainment"]),
            g(&["energy", "J_per_output_token"])
        )
    })
    .collect();

• Why it is safe. The closure captures cfg by shared reference and clones it per run. If it tried to mutate shared state, it would not be Send/Sync and would not compile. A data race is a compile error, not a heisenbug in a nightly job.
• Why order is preserved. collect() on a parallel iterator keeps the input order, so the CSV comes out in rate order however the work was scheduled.
• Measured. Eight 4,000-request runs on this 4-core, 8-thread machine:

Source: examples/results.md in Rust_DES_Kernel

The rows that matter here are the first three: rayon inside one call, or plain Python threads calling a Rust function that releases the GIL, both reach about 3× on four physical cores. Deck 02 explains the GIL rows.

This slide's and the next one's tables were re-measured on 2026-10-03, after the simulator's cost model was corrected: steps no longer charge the whole embedding table, and decode attention includes each new token's attention to itself (deck 10).

The Toolchain: cargo, clippy, rustfmt, criterion

Source: examples/results.md in Rust_DES_Kernel

The bare kernel handles about 30 million events a second; the full serving engine, which does real work per event (cost model, batching, accounting), handles 5 to 6 million.

Testing in Rust: cargo test and proptest

Tests live next to the code (#[cfg(test)] mod tests, which can reach private items) or in tests/ (integration tests, which see only the public API). proptest adds property-based testing: generate inputs, check a property, shrink any failure to a minimal case.

proptest! {
    /// Whatever is scheduled, events come out in (time, priority, insertion) order.
    #[test]
    fn pops_in_time_priority_fifo_order(ops in prop::collection::vec((0u8..20, any::<bool>()), 1..300)) {
        let mut s = Scheduler::new();
        for (k, &(t, urgent)) in ops.iter().enumerate() {
            let p = if urgent { Priority::Urgent } else { Priority::Normal };
            s.schedule_at(t as f64, p, (t, p, k));
        }
        let mut last: Option<(u8, Priority, usize)> = None;
        while let Some((time, e)) = s.pop() {
            prop_assert_eq!(time, e.0 as f64);
            if let Some(l) = last {
                prop_assert!(l < e, "{:?} came out before {:?}", l, e);
            }
            last = Some(e);
        }
        prop_assert_eq!(s.processed() as usize, ops.len());
    }
}

Deck 06 covers the strategy behind these layers, mutation testing (which found gaps here that 95% line coverage did not), and the Python and C++ equivalents.

What to Take Away

• The kernel is the Python kernel with types: BinaryHeap<Reverse<Entry>>, ordered by time (total_cmp), priority and a sequence number.
• Events are enums; the model is a trait; exhaustive match means a new event kind cannot be silently ignored.
• Own state in arenas, pass indices. It satisfies the borrow checker without Rc<RefCell>, and indices can be logged and replayed.
• SimPy processes become state machines, one state per yield, with SimPy's wake-up rules copied exactly.
• Result for expected failures, panic for broken invariants, and validate configuration before the run starts.
• rayon makes sweeps parallel and safe; clippy, rustfmt, proptest and criterion are one command each.

• Rust_DES_KernelAll the code on these slides, with its tests and recorded results.
• Deck 02: Porting a Simulator Core with PyO3Exposing this engine to Python and proving it gives identical answers.
• Interview_RustRust interview questions: ownership, lifetimes, traits, concurrency.
• LLM Inference Simulators 02: Simulator Development TutorialThe same kernel, built step by step in Python, and SimPy's vocabulary.
• The Rust Programming LanguageThe official book; chapters 4 (ownership), 10 (traits) and 16 (concurrency) are the ones this deck leans on.