FHE Accelerator Simulators — Presentation 02

Anatomy of CKKS Bootstrapping

ModRaise, CoeffToSlot (a homomorphic DFT evaluated with baby-step giant-step rotations), EvalMod (a polynomial approximation of modular reduction) and SlotToCoeff, with exact operation counts per stage, where the levels go, and the data-movement profile that makes bootstrapping a memory problem.

ModRaise CoeffToSlot EvalMod SlotToCoeff BSGS Levels
ModRaise → CoeffToSlot → EvalMod → SlotToCoeff
00

Topics We'll Cover

Concepts used here, and where they are explained. Each links to its series glossary entry: a short explanation, then links to the slides that explain it in depth, in this series, the Cryptography decks or LLM Inference Simulators.

01

Why Bootstrap, and the Four Steps

Each multiplication consumes a level (deck 01). When a ciphertext reaches level 0 it cannot be multiplied again. Bootstrapping homomorphically evaluates (an approximation of) decryption to produce an equivalent ciphertext at a high level. For CKKS the standard pipeline is that of Cheon, Han, Kim, Kim and Song (EUROCRYPT 2018, ePrint 2018/153), refined by Han & Ki (ePrint 2019/688) and Bossuat et al. (EUROCRYPT 2021, ePrint 2020/1203):

ModRaiselevel 0 → L0 levels CoeffToSlothomomorphic DFT (BSGS)3 levels, 43 rotations EvalModapproximate x mod q010 levels, 36 HMults SlotToCoeffinverse DFT (BSGS)3 levels, 42 rotations ark set (N = 2^16, L = 23, dnum = 4), full slots: 16 levels spent, 7 left for the application. Counts from the companion trace generator; ARK reports 15 levels for its own bootstrap at these parameters.

The orange stages are rotation-heavy and key-hungry; EvalMod is multiplication-heavy and reuses one relinearisation key. That difference drives everything in decks 03–05.

02

ModRaise

Ordering variants: SlotToCoeff first

SlotToCoeff can run first, on the nearly exhausted input, before ModRaise (OpenFHE's BTSlotsEncoding option). Its rotations then use keys with few limbs, its levels are no longer taken from the top of the chain, and for real-valued data EvalMod runs once instead of twice. In the companion model (option stc_first) the ark-set bootstrap leaves 10 levels instead of 7 and takes 11.51 ms instead of 13.94 ms on the ARK-class design. OpenFHE's own StC-first trace shows the same shape (examples/results.md OpenFHE's own StC-first trace shows the same shape (examples/results.md §19 in the code repo).sect;19 in the code repo).

03

CoeffToSlot: A Homomorphic DFT

Encoding uses a DFT-like map (the canonical embedding). To operate on the polynomial's coefficients, which is where q0·I lives, bootstrapping applies the inverse encoding map homomorphically.

ARK (arXiv:2205.00922) quotes the same structure: with k = 5 over 3 levels, "40 HRots and 158 PMults" per homomorphic DFT. The companion model, without ARK's further optimisations, does 43 and 189.

04

Baby-Step Giant-Step Rotations

A matrix-vector product with d diagonals needs d rotations if done directly. BSGS splits d = n1 × n2: rotate the input n1−1 times (baby steps), form n2 inner sums with pre-rotated diagonals, and rotate each sum once (giant steps).

input x rot(x, 1) rot(x, 2) ⋮ rot(x, 7) 7 baby steps, one shared ModUp Σ pt(0,i) · rot(x,i) Σ pt(1,i) · rot(x,i) ⋮ Σ pt(7,i) · rot(x,i) 8 inner sums: 63 PMults rot(·, 8j) Σ → out 7 giant steps one radix-2^5 level: 14 rotations and 63 PMults instead of 62 rotations
05

EvalMod: Modular Reduction as a Polynomial

After CoeffToSlot the slots hold the coefficients t = m + q0·I (scaled). EvalMod computes t mod q0, which is not a polynomial, so it is approximated: (q0/2π)·sin(2πt/q0) is close to t mod q0 when m is small. In practice this means a scaled cosine evaluated with Chebyshev polynomials, followed by double-angle steps cos(2x) = 2cos2x − 1.

Step (one copy, ark set)HMultsLevels
Scale into the approximation interval (a constant multiply)01
Baby powers T2…T8 (degree 59 → b = 8)73
Giant powers T16, T322(in parallel)
8 baby polynomials (constant multiply-adds), then combine (g = 8)71 + 3
2 double-angle steps22
Total per copy1810
06

SlotToCoeff

07

Where the Levels Go

Parameter setLCoeffToSlotEvalModSlotToCoeffUsed by bootstrapLeft for the application
ark233103167
lattigo243103168
gpu100x3431031618
openfhe-sparse (8 slots)181141162
openfhe-full14 (N = 214)3031432010
08

Operation Counts per Stage

One full-slot bootstrap of the ark set, baseline algorithm (hoisted BSGS). "Requested" bytes are what the operations need, before any on-chip reuse; deck 03 turns them into actual HBM traffic.

StageHE opsHMultHRotPMultDistinct keysNTT + iNTT limbsKey GB requestedPlaintext GB requested
ModRaise10000500.000.00
CoeffToSlot75043189435,5205.222.28
EvalMod55360015,9682.660.00
SlotToCoeff72042189422,1721.410.99
09

The Data-Movement Profile

DFT stages: streaming

  • Every key is used once per bootstrap: compulsory traffic, whatever the SRAM size.
  • Every diagonal plaintext is used once: more compulsory traffic, unless it is generated on chip.
  • Arithmetic per byte is low. On the ARK-class model CoeffToSlot takes 72% of the bootstrap and is HBM-bound (deck 03).

EvalMod: compute

  • One key, reused; ciphertexts are reused across the polynomial evaluation tree.
  • NTT- and base-conversion-heavy. On the same model it takes 10% of the time and its hot-spot is the multiply-add lanes.
  • The 2–3 live ciphertexts and the key fit in a few hundred MiB of SRAM.
The consequence

A bootstrap alternates between a bandwidth-bound phase and a compute-bound phase, so no single roofline point describes it. That is the reason to simulate it as a sequence of operations rather than estimate it from totals. The analytic bound in deck 03 is 28% below the simulated time for exactly this reason.

10

Interactive: Stage Breakdown

"All three" is Min-KS + seeded keys + on-the-fly plaintexts (slide 11). This runs the simulator's trace generator in the browser (the same JavaScript that powers deck 03) and counts operations and requested bytes per stage. No timing here, only the workload.

16
23
4
3
3
2
per stage: limb transforms (green), key GB requested (orange), plaintext GB requested (teal); bars scaled to the largest value in each colour
11

Algorithmic Levers

Each technique changes the workload, not the hardware. Effects on one ark-set bootstrap, from the companion trace generator:

TechniqueWhat it doesEffect on the workload
Hoisting (default)Baby-step rotations share one ModUpFewer NTTs; same keys. Turning it off costs 15% more time on an NTT-starved design (deck 05)
Min-KS (ARK)Apply baby and giant rotations iteratively by one fixed step, so a whole BSGS loop needs only 2 keysCoeffToSlot keys: 43 → 7. Same rotation count, but hoisting is lost (more NTTs) and the rotations are serial
Seeded keysThe uniform half of each evaluation key is regenerated on chip from a seedKey bytes halved; some PRNG work on the multiply-add lanes
On-the-fly plaintexts (in the spirit of ARK's OF-Limb)Generate DFT diagonals on chip from a compact form instead of loading themPlaintext bytes drop to one limb each; adds NTTs
OpenFHE's BSGS (lazy ModDown)Keep rotations in the Q·P basis, multiply plaintexts there, ModDown once per DFT level; then baby steps cost no NTTs, so use many more of themA third fewer transforms (13,710 → 9,234) but 109 rotations instead of 85, and plaintexts stored with the special limbs: right for a CPU, wrong for a memory-bound accelerator (deck 05)
SlotToCoeff firstRun SlotToCoeff before ModRaise, on the nearly exhausted input18 HMults instead of 36 (one EvalMod), 9,544 transforms instead of 13,710, keys 6.95 instead of 9.29 GB requested, and 10 levels left instead of 7. The only lever here that helps every design (deck 05)
Radix / level budgetMore DFT levels → fewer rotationsFewer keys and bytes, fewer useful levels; use the interactive slide

Min-KS, seeded keys and on-the-fly plaintexts trade arithmetic for bytes; OpenFHE's BSGS trades bytes for arithmetic; SlotToCoeff first saves both. Whether that wins depends on whether the design is memory-bound or compute-bound, which is exactly what the simulator in deck 03 decides.

12

Checked Against a Real OpenFHE Bootstrap

OpenFHE v1.5.1 was instrumented (an 82-line patch) to log every NTT, base conversion, key switch, automorphism, rescale and plaintext multiply of a bootstrap. Below is N = 214, 8,192 slots, level budget {3, 3}, dnum 3, against the model at the same parameters, run with the default (balanced) BSGS split and with OpenFHE's. Each cell reads OpenFHE / model / model with OpenFHE's BSGS (HMults: OpenFHE / model).

StageRotationsHMultsNTT + iNTT limbsKey GB requested
CoeffToSlot46 / 35 / 51—2,419 / 5,151 / 2,0251.45 / 1.13 / 1.64
EvalMod—48 / 5620,411 / 11,816 / 11,8161.05 / 1.28 / 1.28
SlotToCoeff45 / 34 / 50—1,025 / 2,494 / 8540.57 / 0.43 / 0.63
13

What to Take Away

Next

Deck 03 builds the accelerator model and runs this workload on it, live.