Rust ↔ Python: Porting a Simulator Core with PyO3

Topics We'll Cover

• Should You Port at All?
• The Shape of a Mixed Rust and Python Project
• PyO3 and maturin in Practice
• What Crosses the Boundary
• Releasing the GIL
• Bit-Exact: The Parity Checklist
• Reproducing Python's Random Numbers
• Three Traps That Cost One ulp
• Testing the Port: Golden and Differential
• Measured: Speed-Ups
• Interactive: What Should You Port?
• Packaging and CI
• What to Take Away

Should You Port at All?

A rewrite is the most expensive way to make a simulator faster. LLM Inference Simulators 08 puts it last for a reason: profile first, then try fewer events, cheaper events and parallel runs. Disaggregated_Inference_Sim's exact fast path already halves the run time in pure Python. Port when what remains is still too slow and the model has stopped changing every week.

The Shape of a Mixed Rust and Python Project

The crucial line is the bottom box: the Rust core never sees a Python type. That keeps it testable with cargo test alone, usable from a Rust CLI, and free to run with the GIL released. maturin's "mixed" layout puts the Python wrapper next to the compiled module:

[tool.maturin]
features = ["python"]
python-source = "python"
module-name = "rust_des._native"

PyO3 and maturin in Practice

PyO3 generates the CPython glue from attributes on ordinary Rust functions; maturin builds and installs the result as a wheel. Argument and return types are converted automatically: a Python list of tuples arrives as Vec<(f64, i64, i64)>.

/// `poisson_workload` with Python's own random stream: the same rows as
/// `disagg_sim.workload.poisson_workload` for the same arguments.
#[pyfunction]
#[pyo3(signature = (rate, n, prompt_mean, prompt_cv, output_mean, output_cv, seed=0))]
fn poisson_workload(
    rate: f64,
    n: usize,
    prompt_mean: f64,
    prompt_cv: f64,
    output_mean: f64,
    output_cv: f64,
    seed: u64,
) -> Vec<Row> {
    disagg::poisson_workload(
        rate,
        n,
        LengthDist::new(prompt_mean, prompt_cv),
        LengthDist::new(output_mean, output_cv),
        seed,
    )
    .into_iter()
    .map(|r| (r.arrival, r.prompt_len, r.output_len))
    .collect()
}

/// Rust core of rust_des: a bit-exact port of Disaggregated_Inference_Sim's engine.
#[pymodule]
fn _native(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(simulate_json, m)?)?;
    m.add_function(wrap_pyfunction!(simulate_many_json, m)?)?;
    m.add_function(wrap_pyfunction!(poisson_workload, m)?)?;
    m.add("__version__", env!("CARGO_PKG_VERSION"))?;
    Ok(())
}

pip install maturin
maturin develop --release -E dev    # compile, install into the venv, plus test extras
pytest pytests                      # Python-side tests import rust_des like any package
maturin build --release             # a wheel for distribution

What Crosses the Boundary

Every value that crosses from Python to Rust is converted, and every call has a fixed cost. So the boundary should be coarse (one call per simulation, never one per event) and plain (numbers, tuples, strings; no Python objects held by Rust).

The wrapper times each part of a call, so the cost of the boundary is measured, not guessed:

Source: examples/results.md in Rust_DES_Kernel

Two lessons. The conversion itself is small (under a millisecond for 1,000 requests). But the first version left the summary in Python, and Python's summarise() takes far longer than the simulation it summarises: about 50 ms against 4 ms (about 100 ms before it learned to sort once; deck 11). Port what profiling says is slow, and that includes the code around the engine. The summary is the biggest Rust cost too, because it sorts and sums every inter-token latency: one per output token.

Releasing the GIL

CPython's global interpreter lock (GIL) lets one thread run Python bytecode at a time. Rust code that touches no Python objects does not need it, and can say so. Then other Python threads keep running, including other calls into the same simulator.

/// Run one simulation. Returns (per-request stamps, summary JSON, events processed,
/// seconds simulating, seconds summarising).
#[pyfunction]
#[pyo3(signature = (config_json, rows, with_summary=true))]
fn simulate_json(
    py: Python<'_>,
    config_json: &str,
    rows: Vec<Row>,
    with_summary: bool,
) -> PyResult<Output> {
    let spec = spec(config_json)?;
    // The GIL is released for the whole run: nothing in here touches Python.
    py.detach(|| run_one(&spec, &rows, with_summary))
        .map_err(PyValueError::new_err)
}

• The compiler enforces it. The closure passed to detach must be Send and cannot use the Python token, so it cannot touch a Python object by mistake. That is why the inputs are converted to plain Rust data first.
• Two ways to use it: ordinary Python threads calling rust_des.simulate, or one call to simulate_many, which runs a batch on rayon's thread pool.

Source: examples/results.md in Rust_DES_Kernel

SimPy holds the GIL, so four threads are slower than one (they contend for the lock); only processes help it. The Rust rows reach about 3× on this 4-core, 8-thread machine. On the free-threaded build of CPython there is no GIL to release; PyO3 0.28 and later declare modules safe for it by default (gil_used = true opts out), but an abi3 wheel does not cover that build, which has its own ABI (PyO3 guide).

Bit-Exact: The Parity Checklist

"Agrees to within a tolerance" hides bugs: a wrong tie-break or an off-by-one in batching changes results by less than the tolerance on most workloads. Identical answers rule out everything at once, and they are achievable, because IEEE 754 arithmetic is deterministic. What makes two programs differ is doing different operations. Everything that had to match:

Reproducing Python's Random Numbers

A port that cannot generate the same workload can only be tested on workloads passed in from Python. Reproducing random.Random removes that limit, and it is less work than it sounds: CPython uses the Mersenne Twister (MT19937), seeds it with init_by_array, and builds the variates in pure Python.

/// `random()`: a float in [0, 1) with 53 random bits.
pub fn random(&mut self) -> f64 {
    let a = (self.next_u32() >> 5) as f64;
    let b = (self.next_u32() >> 6) as f64;
    (a * 67_108_864.0 + b) * (1.0 / 9_007_199_254_740_992.0)
}

/// `normalvariate(mu, sigma)`: Kinderman and Monahan's ratio-of-uniforms method.
pub fn normalvariate(&mut self, mu: f64, sigma: f64) -> f64 {
    let nv_magicconst = 4.0 * (-0.5f64).exp() / 2.0f64.sqrt();
    loop {
        let u1 = self.random();
        let u2 = 1.0 - self.random();
        let z = nv_magicconst * (u1 - 0.5) / u2;
        let zz = z * z / 4.0;
        if zz <= -u2.ln() {
            return mu + z * sigma;
        }
    }
}

• Seeding. An integer seed is split into 32-bit words, least significant first, and fed to init_by_array, which is why Random(5) and Random(2**40 + 7) both work.
• Tested against CPython with values printed by Python 3.12, and end to end: the Rust generator emits exactly the rows of poisson_workload for four seeds and 500 requests each (test_workload_generator_reproduces_python_random).
• Lognormal lengths also need round() (half to even, not Rust's default round, which rounds halves away from zero) and Python's clamp order.

Three Traps That Cost One ulp

Each of these produced, or would have produced, a difference in the last bit of some result. Each was found by a test, not by reading the code.

fn main() {
    let (mut differ, mut total) = (0u32, 0u32);
    let mut x = 1e-3_f64;
    while x < 1e6 {
        total += 1;
        if x.cbrt() != x.powf(1.0 / 3.0) {
            differ += 1;
        }
        x *= 1.000123; // a geometric sweep over nine decades
    }
    println!("{differ} of {total} values differ");
    println!(
        "0.001: cbrt {:?}, powf(1/3) {:?}",
        0.001f64.cbrt(),
        0.001f64.powf(1.0 / 3.0)
    );
}

100783 of 168493 values differ
0.001: cbrt 0.1, powf(1/3) 0.10000000000000002

Testing the Port: Golden and Differential

Two complementary checks, both demanding identical timestamps and an identical summary dictionary:

def assert_identical(py, rs):
    for j, name in enumerate(STAMPS):
        a = [getattr(r, name) for r in py.requests]
        b = [s[j] for s in rs.stamps]
        if a != b:
            k = next(i for i, (x, y) in enumerate(zip(a, b)) if x != y)
            pytest.fail(f"{name} differs first at request {k}: python {a[k]!r} rust {b[k]!r}")
    assert normalise(summarise(py)) == rs.summary

Source: examples/results.md in Rust_DES_Kernel

When a difference appears, the message names the first request and stamp that differ, which usually points straight at the event-order rule involved.

Measured: Speed-Ups

Source: examples/results.md in Rust_DES_Kernel

• 71 to 74× over SimPy, 36× over the Python fast path, for the engine alone. Through PyO3, including the Rust summary and the conversions, a 1,000-request run takes about 13 ms instead of 0.33 s (SimPy plus summarise()).
• Events per second are not comparable one to one: SimPy also processes process start-ups, resource grants and releases, so it handles more events for the same model.
• Speed-ups are noisy on a desktop. The first recording gave 53 to 69×. Three runs on 2026-10-03, after the cost-model correction (deck 10), gave 65 to 76×. Neither the correction nor the summary change touches the event loop. Quote a range, and see deck 11 for why a single timing is not a number.

Interactive: What Should You Port?

Amdahl's law with measured parts. Choose a workload and decide where each stage runs. Times come from examples/results.md; the boundary cost applies whenever Rust is involved. LLM Inference Simulators 08, slide 14 has the general version.

Packaging and CI

• One wheel for every Python. PyO3's abi3-py310 feature builds against the stable ABI, so one wheel per platform serves CPython 3.10 and later.
• The core builds without Python. The PyO3 code is behind a python feature that maturin switches on; cargo test and the CLI never link against Python.
• CI runs both halves. One job: rustfmt, clippy, cargo test (including the golden files) and a CLI smoke test. The other: clippy on the bindings, maturin develop, and the differential tests with the Python reference installed from its repository.

python:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - uses: dtolnay/rust-toolchain@stable
      with:
        components: clippy
    - uses: Swatinem/rust-cache@v2
    - uses: actions/setup-python@v5
      with:
        python-version: "3.12"
    - name: Clippy (PyO3 bindings)
      run: cargo clippy --features python -- -D warnings
    - name: Build the extension and install the Python reference simulator
      run: |
        python -m venv .venv
        .venv/bin/pip install maturin
        .venv/bin/maturin develop --release -E dev
    - name: Differential tests against Disaggregated_Inference_Sim
      run: .venv/bin/pytest pytests

What to Take Away

• Port last, port the hot path, keep the reference. The Python model is what makes the Rust trustworthy.
• Keep the Rust core free of Python types; make the boundary coarse and plain; measure what it costs.
• Release the GIL with py.detach (formerly allow_threads): Python threads then run simulations in parallel.
• Aim for bit-exact, not "close": copy operation order, event order, fsum, rounding, the random stream and the exact expressions.
• Test it two ways: golden files that need no Python, and a live differential suite with generated configurations.
• Profile the whole call: here the summary, not the engine, was the next bottleneck.

• Rust_DES_KernelThe port, its bindings, its tests and examples/results.md.
• The PyO3 user guide and maturin documentationCheck them for the version you use; names changed in 0.26.
• Deck 06: Testing Frameworks for SimulatorsGolden, differential, property-based and mutation testing in depth.
• LLM Inference Simulators 08: Accelerating SimulatorsEverything to try before a rewrite.
• Interview_RustRust interview questions.