Role Primer — Simulation & Frameworks Engineer

Every rotation and relinearisation needs a key switch with its own evaluation key, tens to hundreds of MB at N = 2¹⁶, so streaming keys from HBM dominates. Fixes: reuse keys across rotations (Min-KS, hoisting), generate half of each key on chip from a seed, generate plaintexts on the fly, reorder the bootstrap (SlotToCoeff first), and size the scratchpad so keys stay on chip. In the sister series these move the bottleneck from HBM to the MAC lanes. (FHE 01, 05.)

Check operation counts against analytical formulas per bootstrap stage; record the real kernel stream from an instrumented library (OpenFHE) and compare structure; replay that trace on a CPU-like configuration and compare with measured time; run compiler-generated programs (HEIR) and require the same rotations and keys as the compiler's own library output; keep a bit-exact twin for differential tests. (FHE 03; deck 06.)

Only when the design is NTT-bound: on a memory-bound configuration a faster transform moves nothing. Exact modular NTTs must be mapped onto analogue transforms by splitting coefficients into digits, which multiplies conversions, and every conversion costs energy that grows with resolution. Answer with the bound attribution first, then the precision and conversion budget. (FHE 04; deck 07.)

HEIR lowers a secret-annotated program to a CKKS-dialect program with explicit rotations, relinearisations, rescales and bootstraps; a front end reads that and emits the simulator's operation trace. Cost feedback into HEIR's choices (for example where to place bootstraps) is the natural next step. (FHE 03; deck 09.)

DMA and compute as processes; the FIFO as simpy.Store(capacity=k); memory channels as a Resource. yield fifo.put(tile) blocks when full, which is the back-pressure. Measure DMA stall time on put and compute stall on get; sweep k to find where throughput saturates. (Deck 02, slide 07.)

• Verification ladder: unit, invariants, analytic checks (queueing results, Little's law), behavioural expectations.
• Validate the method on known hardware first: model a GPU, compare with measured vLLM or Vidur numbers.
• Correlate against RTL or emulation block by block as they become available, and track the correlation error as a metric.
• State assumptions explicitly and run sensitivity sweeps on the uncertain parameters. (Decks 01 and 06.)

Different questions (what to build versus did we build it right) at different times. The model is the executable spec and golden reference (scoreboards via DPI-C or cocotb); RTL cycle counts calibrate the model; workload traces become directed tests; co-simulation mixes RTL islands into the system model. (Deck 01, slides 06–08.)

Profile. Reduce events (coarser granularity, macro-steps). Make events cheaper (incremental state, lazy bookkeeping, hoisting). Cut probe cost. Parallelise across runs. Do fewer runs (analytic bounds, bisection, Bayesian optimisation, common random numbers). Only then a compiled core or PDES. Prove each step exact with differential tests. (Deck 08.)

Same throughput and utilisation; different latency distributions (10/20 ms against 20/20 ms for two 10 ms transfers). It matters for tail metrics. Choose from the hardware's actual arbitration, document it, and test it. (Deck 02, slide 08.)

Batched resources look 100% busy at low efficiency. Use a stage breakdown that sums to end-to-end latency, take the largest waiting stage, map it to its owning resource, and confirm with a what-if sensitivity run. (Deck 06, slide 04.)

Each step reads every weight and all the KV cache to produce one token per sequence: arithmetic intensity is about the batch size, far below the ridge point. Faster matmul alone barely helps decode; the design must address memory bandwidth, capacity and data movement. Prefill is compute-bound and benefits directly. (Deck 03.)

Start with a TorchDispatchMode trace on the meta device (cheap, any model); then torch.export graphs for the compiler path; a torch.compile backend for one-line user adoption; a PrivateUse1 device for functional bring-up. Track operator coverage as a metric. (Deck 09.)

The number-theoretic transform is the FFT over a finite field. FHE ciphertexts are polynomials of degree 2¹³–2¹⁷ in RNS form, and polynomial multiplication, key switching and bootstrapping all run through NTTs, so NTT throughput, together with the memory traffic of ciphertexts and keys, dominates FHE cost. (Deck 09.)

HEIR is an MLIR-based FHE compiler: secret-annotated programs are lowered to scheme dialects (CKKS, BGV, CGGI) and then to polynomial and modular arithmetic, then to backends. A hardware team lowers into its own dialect and emits an operation stream (NTTs, modular arithmetic, key switches) for the simulator, using a CPU backend such as OpenFHE as the functional oracle. (Deck 09.)

Price the counts the simulator already has: static power × time, energy per FLOP or conversion, per byte of memory traffic, per bit on links. Add DVFS and power caps as a third roof on step time. Report average and peak power, joules per token and the static/compute/memory/link split. Calibrate coefficients by regression on measured power (DCGM) or from RTL power analysis driven by the simulator's workload traces. Use it to find free savings (memory-bound phases), the energy-optimal cap, and how load affects energy per token through static power. (Deck 07.)

• Correlation error against RTL or emulation per workload, with a target.
• Operator and model coverage for each framework path.
• Test pass rate, coverage, and golden-metric stability.
• Simulator speed (simulated seconds per wall-clock second).
• Open accuracy issues by severity; milestone burn-down in Jira. (Deck 06.)

Configuration as data; seeds per component; outputs stamped with config hash and git SHA; deterministic tie-breaking; warm-up deletion; replications with confidence intervals; common random numbers for comparisons; CI that re-runs golden configurations. (Decks 02 and 06.)