Simulation Engineering Toolkit 13 — Power, Performance and Area: the Architect's Trade-offs

00

Topics We'll Cover

What PPA Is, and Who Trades It
Performance: Latency and Throughput
Power: Dynamic, Static, DVFS and Dark Silicon
Area: What Sets It
Estimating Area Before Layout
Why Area Is Cost: Dies per Wafer and Yield
Composite Metrics: perf/W, perf/mm², EDP and TCO
Pareto Fronts: Why There Is No Single Best Design
Interactive: PPA Explorer
Worked Example: The Scratchpad Sweep
Compute or SRAM, and When to Split the Die
The Other Trade-offs, Named
Pitfalls
What to Take Away

Concepts used here, and where they are explained. Each links to a glossary entry: this series glossary, or the glossaries of LLM Inference Simulators and FHE Accelerator Simulators for concepts those series already explain.

PPA: power, performance and area TTFT, TPOT and inter-token latency Little's law and the operational laws NTT-, MAC-, memory- and power-bound Roofline, arithmetic intensity and ridge point Dynamic power (αCV²f) and leakage Static and dynamic power DVFS and power caps (the third roof)TDP and peak power Dynamic power manager against worst-case clocking Dark silicon What sets die area CACTI McPAT Accelergy and Timeloop An area model calibrated from papers and CACTI Silicon area and the area model Dies per wafer Poisson yield Murphy yield Die yield and silicon cost Reticle limit and chiplets perf/W perf/mm² and perf/$EDP and ED²P Total cost of ownership (TCO)Energy per token and energy–delay trade-off Pareto front and dominance Design sweeps and Pareto fronts Scratchpad, LRU and spills Min-KS, seeded keys and on-the-fly plaintexts Continuous batching Disaggregated (prefill/decode) serving ENOB and exact rounding The fidelity ladder

01

What PPA Is, and Who Trades It

PPA is power, performance and area: the three quantities every chip design is judged on. The simulation series in this GitHub measure performance and power carefully; this deck adds the third axis, and names the trade-offs between all three.

Who	Decides	Trades, for example
Architecture	Which units, how many, how much on-chip memory, which memory technology, which algorithms the hardware is built for	More SRAM cuts off-chip traffic (performance, energy) and costs area
Micro-architecture	Pipelining, banking, datapath widths, the clock target, how units share ports and wires	A deeper pipeline raises the clock and costs registers (area, power)
Physical design	Floorplan, cell libraries and threshold voltages, SRAM bitcell choice, wire layers, power grid	Low-leakage cells cut static power and are slower

The three cannot be optimised separately. Power depends on the clock, which performance wants high. A bigger die has longer wires, which are slower and burn more energy per bit. On-chip memory saves off-chip energy and costs leakage and area. Each decision moves at least two axes.
Area matters because it is cost. Wafers are bought by the square millimetre, and yield falls as dies grow (slide 06), so a design twice the size costs more than twice as much.
Early decisions matter most. Architecture fixes most of the PPA envelope before any RTL exists, which is why an architect needs a simulator that reports all three axes, not just latency.

02

Performance: Latency and Throughput

Performance is two different quantities, and a design can win one while losing the other:

Latency: the time one task takes (one bootstrap, one token, one request), reported as a distribution with its tail, not an average.
Throughput: tasks per second when the machine is kept busy. Little's law links the two: concurrency = throughput × latency.
Batching, pipelining and more units in parallel raise throughput and often raise latency; the metrics users feel (time to first token, time per output token, goodput under an SLO) are the ones a product is judged on.

The metrics, their statistics and how to attribute them to hot-spots are covered in InfSim 06; this deck does not repeat them. Two points matter here:

A PPA comparison needs one performance number per design, chosen for the product: bootstraps per second for an FHE accelerator, tokens per second at a latency SLO for an LLM server.
The bound tells you which resource to buy. A memory-bound design gains nothing from more compute (slide 11 measures this): see bound attribution and the roofline.

03

Power: Dynamic, Static, DVFS and Dark Silicon

The two kinds of power

Dynamic: P_dyn = α C V² f, for activity factor α (the fraction of the capacitance C switched each cycle), supply voltage V and clock f. Static (leakage): P_static = V · I_leak, drawn whenever the circuit is powered, busy or not.

DVFS (dynamic voltage and frequency scaling) lowers f and V together. Over the range where f rises roughly with V, dynamic power grows roughly as f³ and energy per operation as V², so running slower saves energy per task, until static power, charged for the longer run time, eats the saving. InfSim 07 models DVFS and power caps; FHESim 05 shows a dynamic power manager against worst-case clocking.
TDP (thermal design power) is the heat the package and cooling can remove. A design whose units could together draw more must throttle: power becomes a limit on performance (a "power-bound" verdict in the FHE simulator).
Dark silicon. Since supply voltage stopped scaling with feature size (the end of Dennard scaling), each new node gives more transistors than the power budget can switch at once; part of the chip must be idle or slowed at any moment (Esmaeilzadeh et al., ISCA 2011). Specialised accelerators are one response: area that is dark most of the time but efficient when lit.
Energy is not power. Power limits the package; energy per task sets battery life and the electricity bill. A faster design at higher power can use less energy per task ("race to idle") when static power dominates.
Leakage is chosen, not given. CACTI models the same 512 MiB scratchpad leaking 0.14 W / 179 W with low-standby-power against high-performance transistors (22 nm; CACTI card). Large memories use low-leakage cells and give up speed.

04

Area: What Sets It

Four things take the silicon: SRAM bitcells (with their decoders and sense amplifiers), logic (here modular multipliers, which dominate FHE datapaths), I/O and PHYs (the analogue interfaces to HBM, PCIe, die-to-die links) and wiring, with the repeaters and pipeline registers long wires need. A published breakdown shows the proportions: ARK, an FHE accelerator modelled at 7 nm (Kim et al., MICRO 2022, Table IV), reproduced by the area model in FHE_Accelerator_Sim:

design	NTT	MAC	permute	SRAM	uncore	HBM PHY	die mm²
ARK as published (8,192 bfly, 1,024 perm. words/cycle)	57.2	18.2	20.6	229.2	63.4	29.6	418.2
ARK-class (this model's default)	28.6	18.2	82.4	229.2	69.9	29.6	457.9
small digital	3.6	4.5	20.6	229.2	50.3	59.2	367.4
small + hybrid optical*	3.6	4.5	20.6	229.2	50.3	59.2	368.9
small + ideal optical*	3.6	4.5	20.6	229.2	50.3	59.2	368.9

Source: examples/results.md in FHE_Accelerator_Sim

SRAM is the largest item: 229.2 mm² of the 418.2 mm² die, for 512 MB of scratchpad.
Wires are hidden inside the units. ARK reports that the logic of its four NTT units is 34.9 of their 57.2 mm², and of its four automorphism units 0.9 of 20.6 mm²; the rest is wiring between lanes. Permutation networks are almost all wire.
PHYs do not shrink with the node the way logic does, and their count is set by the bandwidth wanted: two HBM stacks are 29.6 mm² in ARK and in BTS.
The second row is this simulator's default design. Its permutation network moves 4× ARK's words per cycle, so it costs 82.4 mm² for a unit the simulator shows 2% busy. Nobody noticed until area was modelled: that is the point of the third axis.

05

Estimating Area Before Layout

Before RTL, area comes from analytical models; after RTL, from synthesis; only layout gives the real number. Three pre-RTL estimators are explained, with their accuracy, on SimEng 12's cards: CACTI (SRAM and caches), McPAT (processors) and Accelergy and Timeloop (accelerators, from component tables). FHE_Accelerator_Sim's area model (ppa.py) is built the same way, from three sources:

Units: ARK's per-unit areas (above) divided by its unit counts: about 0.007 mm² per NTT butterfly and 0.002 mm² per multiply-add lane, at 7 nm, wiring included. Scaling linearly to other counts is the illustrative part: wiring grows faster than the units it joins.
SRAM: CACTI 7, built and run here, gives the shape of area against capacity for banked scratchpads (22 nm is its smallest node). One factor scales it to 7 nm so that 512 MiB matches ARK's scratchpad; BTS's 512 MB agrees within 3%.
Everything else: HBM PHYs per stack, and register files plus on-chip network as ARK's fraction of the rest.

MiB	22 nm mm²/MiB (CACTI)	array efficiency	7 nm mm²/MiB (model)	7 nm mm² (model)
64	0.9189	72.6%	0.4688	30.0
128	0.8901	74.9%	0.4541	58.1
256	0.9020	73.9%	0.4602	117.8
512	0.8774	76.0%	0.4477	229.2
1024	0.8189	81.4%	0.4178	427.8
2048	0.7995	83.4%	0.4079	835.4

Source: examples/results.md in FHE_Accelerator_Sim

CACTI says area per MiB falls slightly as a banked scratchpad grows (array efficiency 72.6% at 64 MiB, 83.4% at 2 GiB), so the model follows its curve instead of using a constant. These are model outputs, not silicon.
Every coefficient is labelled with its node and source in the code, and the area never feeds back into the simulated time or energy: the 104 tests that existed before the area model still pass unchanged.

06

Why Area Is Cost: Dies per Wafer and Yield

Three formulas

Dies per wafer (wafer diameter d, die area A): π(d/2)²/A − πd/√(2A); the second term is the partial dies lost round the edge. Poisson yield: Y = e^−A·D₀ for defect density D₀, if defects land independently and uniformly. Murphy yield (Murphy, 1964): Y = ((1 − e^−A·D₀) / (A·D₀))², for a defect density that varies across the wafer. Cost per good die = wafer price / (dies per wafer × Y).

Defect density D₀

Die area

Cost grows faster than area. With the illustrative D₀ = 0.1 per cm² and $10,000 per wafer, a good square millimetre costs 2.54x as much on an 858 mm² die as on a 100 mm² one (results.md §24).
The two models agree for small dies and part for large ones, because Murphy's varying density leaves some large dies defect-free. Raise D₀ to see them diverge; at 0.5 per cm² Murphy's yield is 1.52× Poisson's for the FHE die. Which one a foundry's data fits is an empirical question.
The reticle sets a hard limit: a lithography field is 26 × 33 mm (858 mm²), so no single die can be larger. Past that, or well before it for cost, designs split into chiplets (slide 11).

07

Composite Metrics: perf/W, perf/mm², EDP and TCO

Combining axes into one number makes designs comparable, and each combination hides something. With perf = tasks per second, t = time per task and E = energy per task:

Metric	Formula	Rewards	Hides
perf/W	(1/t) / (E/t) = 1/E	Energy per task, and nothing else	Speed: a slow design with the same energy scores the same
perf/mm²	1 / (t · A)	Throughput per unit of silicon (cost)	Energy, and yield (a mm² on a big die costs more)
perf/$	(1/t) / cost per good die	Throughput per unit of manufacturing cost	Running cost, and everything but silicon
EDP	E · t	Energy and speed equally	Area
ED²P	E · t²	Speed more than energy; roughly independent of supply voltage where f rises with V (E ∝ V², t ∝ 1/V), so it compares designs, not operating points	Area
TCO	purchase cost amortised over the service life + energy × electricity price × datacentre overhead	The money an operator actually spends	Nothing, but it needs prices you rarely have pre-silicon

Which product optimises which: a phone SoC lives on energy per task and area (perf/W, perf/$); a datacentre accelerator on throughput per TCO dollar, of which power is a large share; a supercomputer node on performance under a power cap; an edge device on cost and standby power.
perf/W equals 1/energy. The FHE simulator's default design scores 0.88 bootstraps per joule with the baseline algorithm, which is just 1 / 1139 mJ. Quoting both as if they were separate wins counts the same thing twice.
For energy per token and the energy–delay curve in LLM serving, see InfSim 07.

08

Pareto Fronts: Why There Is No Single Best Design

Design A dominates design B if A is no worse on every axis and better on at least one. The Pareto front is the set of designs nothing dominates. Every design off the front can be improved for free; every design on it is a different compromise, and choosing among them needs the product's priorities, not more simulation.

Adding an axis adds designs to the front. Two designs with equal latency and energy are both on a latency–energy front; the area axis separates them. In the FHE simulator's baseline scratchpad sweep, latency and energy both fall as SRAM grows, so on those two axes the largest scratchpads win outright; with area as a third axis, every size up to 2 GiB is on the front (slide 10). Its 36-design sweep without area has a single optimum (FHESim 05).
Finding the front is cheap once the points exist: search.pareto_nd compares every pair (O(n²), fine for hundreds of designs). Finding the points is the expensive part, which is why smarter experiments (bounds, bisection, surrogates) matter.

09

Interactive: PPA Explorer

Each design is simulated in your browser by FHE_Accelerator_Sim's JavaScript port (bit-exact with the Python package) running one full-slot ark-set bootstrap, then priced by the ported area model. Grey points are a grid of 80 designs; green ones are on the three-way (latency, energy, area) Pareto front; the ring is the design the controls select. The plot shows two of the three axes, so a green point can look dominated here and still win on energy (hover for its numbers). Area coefficients are 7 nm and illustrative; the node controls scale them.

Algorithm

Scratchpad

NTT butterflies/cycle

MAC lanes

SRAM density factor

Logic density factor

D₀ per cm²

Try: the baseline algorithm at 2× NTT and MAC (memory-bound: the extra compute moves the ring right, not down); the all-techniques design at 256 MiB against 512 MiB; the SRAM density factor at 2× (the front shifts toward small scratchpads).
The defaults (all techniques, 512 MiB, 4,096 butterflies, 8,192 lanes) reproduce results.md §22: 7.19 ms, 607 mJ, 457.9 mm². The deck's browser test checks these readouts against results.md.

10

Worked Example: The Scratchpad Sweep

FHE_Accelerator_Sim's scratchpad sweep, priced in area (results.md §22). With the baseline algorithm every rotation key is used once per bootstrap, so the design stays memory-bound and SRAM keeps buying latency:

SRAM MiB	bootstrap	mJ	die mm²	perf/mm² (1/s/mm²)	perf/W (1/J)	EDP (mJ·s)	Pareto
128	26.98 ms	2084	253.5	0.146	0.48	56.22	yes
256	17.10 ms	1383	324.8	0.180	0.72	23.65	yes
384	14.57 ms	1193	392.3	0.175	0.84	17.39	yes
512	13.94 ms	1139	457.9	0.157	0.88	15.87	yes
768	13.33 ms	1096	581.1	0.129	0.91	14.60	yes
1024	11.69 ms	984	695.2	0.123	1.02	11.50	yes
2048	11.41 ms	966	1182.3 (> reticle)	0.074	1.04	11.02	yes
4096	11.41 ms	966	2180.5 (> reticle)	0.040	1.04	11.02

Source: examples/results.md in FHE_Accelerator_Sim

With Min-KS, seeded keys and on-the-fly plaintexts the key traffic is gone by 512 MiB and the design turns MAC-bound:

SRAM MiB	bootstrap	mJ	die mm²	perf/mm² (1/s/mm²)	perf/W (1/J)	EDP (mJ·s)	Pareto
128	21.22 ms	1691	253.5	0.186	0.59	35.88	yes
256	8.71 ms	774	324.8	0.354	1.29	6.74	yes
384	7.56 ms	646	392.3	0.337	1.55	4.88	yes
512	7.19 ms	607	457.9	0.304	1.65	4.37	yes
768	7.19 ms	607	581.1	0.239	1.65	4.37
1024	7.19 ms	607	695.2	0.200	1.65	4.37
2048	7.19 ms	607	1182.3 (> reticle)	0.118	1.65	4.37
4096	7.19 ms	607	2180.5 (> reticle)	0.064	1.65	4.37

Source: examples/results.md in FHE_Accelerator_Sim

Each metric picks a different winner. With all techniques, perf/mm² peaks at 256 MiB (0.354), latency and energy bottom out from 512 MiB, and EDP never improves past 512 MiB while area keeps growing. The Pareto front is 128 to 512 MiB; past that, more SRAM is all cost.
The algorithm changes the hardware answer. The baseline algorithm wants SRAM up to the reticle; the co-designed one wants 256 to 512 MiB. This is the area version of FHESim 05's finding that "how much SRAM" depends on the algorithm.
The large rows are not buildable: above about 2 GiB of SRAM the die passes the reticle limit, and 4 GiB is dominated anyway (same speed, twice the silicon).

11

Compute or SRAM, and When to Split the Die

From a 256 MiB design, one change at a time; "ms saved per 100 mm²" is the return on the area spent (§23). Baseline algorithm (memory-bound):

design	die mm²	area added	bootstrap	mJ	perf/mm²	ms saved per 100 mm²
start: 256 MiB, 4,096 bfly, 8,192 MAC	324.8	+0.0	17.10 ms, memory-bound	1383	0.180	-
2x NTT + MAC	380.7	+55.9	16.98 ms, memory-bound	1379	0.155	0.21
4x NTT + MAC	492.5	+167.8	16.96 ms, memory-bound	1378	0.120	0.08
2x MAC only	346.5	+21.7	17.14 ms, memory-bound	1385	0.168	-0.20
+128 MiB SRAM (384)	392.3	+67.5	14.57 ms, memory-bound	1193	0.175	3.74
+256 MiB SRAM (512)	457.9	+133.1	13.94 ms, memory-bound	1139	0.157	2.38
permutation network cut to 1,024 words/cycle	250.9	-73.8	17.09 ms, memory-bound	1383	0.233	-

Source: examples/results.md in FHE_Accelerator_Sim

All three techniques (MAC-bound):

design	die mm²	area added	bootstrap	mJ	perf/mm²	ms saved per 100 mm²
start: 256 MiB, 4,096 bfly, 8,192 MAC	324.8	+0.0	8.71 ms, MAC-bound	774	0.354	-
2x NTT + MAC	380.7	+55.9	7.77 ms, MAC-bound	736	0.338	1.68
4x NTT + MAC	492.5	+167.8	7.63 ms, MAC-bound	731	0.266	0.64
2x MAC only	346.5	+21.7	8.64 ms, MAC-bound	771	0.334	0.34
+128 MiB SRAM (384)	392.3	+67.5	7.56 ms, MAC-bound	646	0.337	1.71
+256 MiB SRAM (512)	457.9	+133.1	7.19 ms, MAC-bound	607	0.304	1.14
permutation network cut to 1,024 words/cycle	250.9	-73.8	8.74 ms, MAC-bound	775	0.456	-

Source: examples/results.md in FHE_Accelerator_Sim

Buy the bound resource. Memory-bound, 128 MiB of SRAM returns 3.74 ms per 100 mm² against 0.21 for doubling the compute: the area cost of more compute loses to buying SRAM. MAC-bound, the two are even.
Removing area can be the best move: cutting the over-provisioned permutation network raises perf/mm² more than any addition, at almost no latency cost.

Splitting a die that is too big (§24): the 2 GiB design as equal chiplets, silicon only.

chiplets	mm² each	fits the reticle	Murphy yield each	silicon $ for the set
1	1182.3	no	34.4%	$727
2	591.1	yes	57.0%	$381
4	295.6	yes	75.0%	$267
8	147.8	yes	86.4%	$219

Source: examples/results.md in FHE_Accelerator_Sim

Chiplets are the standard answer to dies past the reticle or past the yield knee: smaller dies yield better, and only good ones are assembled. This table leaves out what chiplets pay for that (die-to-die PHYs and their area and energy, an interposer or bridge, assembly yield, testing dies before assembly), so it overstates the saving; treat it as the direction, not the size.
The optical engine on the same plane (§25, speculative areas): the realistic hybrid engine scores 0.005 perf/mm² against 0.156 without it, and the ideal engine helps only the NTT-starved design. The photonic die is a placeholder; the question, not the answer, is the point (FHESim 04).

12

The Other Trade-offs, Named

The simulation series make several trade-offs without calling them that. Each is a front with two axes and no single best point:

Trade-off	The lever	What it shows	Where
Latency against throughput	Batch size	Bigger batches fill the hardware and raise throughput; every request in the batch waits longer	InfSim 03 · batching, InfSim 05 · results
Time to first token against time per output token	Disaggregating prefill and decode	Separate pools stop prefill bursts stalling decode, at the cost of moving the KV cache and of hardware that cannot be shared	InfSim 05 · the interference problem, the idea
Precision against energy	ENOB of an optical engine's converters	Each extra bit doubles converter energy; too few bits forces narrower digits and more passes	FHESim 04 · the precision tax, conversion energy
Memory against compute	Min-KS, seeded keys, on-the-fly plaintexts	Regenerating data on chip trades HBM traffic for multiplications: on the memory-rich ARK-class design a bootstrap falls from 13.94 ms to 7.19 ms; on the NTT-starved design it rises from 17.46 ms to 26.65 ms	FHESim 02 · algorithmic levers, FHESim 05 · acceleration techniques
Fidelity against simulation speed	The model's level of abstraction	Each rung down the fidelity ladder is slower and more exact; loosely timed SystemC models trade timing error for speed through the quantum	InfSim 01 · the fidelity ladder, InfSim 08 · choose the abstraction, SimEng 03 · the quantum trade-off

The same lever can move a design either way. The memory–compute row is the sharpest example in the series: the techniques that make the ARK-class design fast make the NTT-starved design slow, because they spend the resource it lacks.
Simulation itself is a PPA problem: a faster, less exact model is a different point on a fidelity–speed front, and the right choice depends on the question.

13

Pitfalls

Optimising one axis. The best perf/W in the baseline sweep (1.04 per joule) belongs to the 2 and 4 GiB designs, which no reticle can print. Report all three axes, or a composite that matches the product, and say which.
Illustrative coefficients presented as truth. The area model's units are one published design's breakdown scaled linearly, its SRAM is a 22 nm CACTI curve scaled by one factor, and its optical areas are placeholders. It is good for ratios, trends and crossovers between designs built from the same assumptions, and wrong as an absolute mm². The explorer's density factors exist to show how far conclusions move when the assumptions do.
Ignoring yield. Pricing area linearly undercounts large dies: at the illustrative D₀, a good mm² on a reticle-sized die costs 2.54x one on a 100 mm² die.
Comparing designs at different nodes without scaling. ARK (418.3 mm² at 7 nm) and CraterLake (472.3 mm² at 12/14 nm) look similar in size (ARK's comparison table); at one node they would not be. Scale to a common node, or compare within one.
Counting the same thing twice. perf/W is 1/energy per task; perf/mm² and perf/$ both reward small dies. A table of five metrics may hold two independent facts.
Forgetting what the model leaves out: here HBM stacks, packaging, test and the system around the chip, none of which the silicon cost includes.

14

What to Take Away

Power, performance and area are traded together at every level from architecture to layout; a simulator that reports only latency answers a third of the question.
Area is cost, and cost grows faster than area through dies per wafer and yield, up to the reticle, past which the design must be split into chiplets.
Pick the composite metric that matches the product, and know what it hides: perf/W ignores speed, perf/mm² ignores energy, EDP and ED²P ignore area.
Pareto fronts replace "the best design" with the set worth choosing between; an extra axis can turn a single optimum into a front.
Buy the bound resource. On a memory-bound FHE design, SRAM returned 3.74 ms per 100 mm² and compute 0.21; and the algorithm changes which resource is bound.
Label every coefficient with its node and source, and say which are illustrative. The conclusions here are about shapes and crossovers, not absolute square millimetres.

Re-running this deck's numbers

Every number comes from examples/results.py in FHE_Accelerator_Sim (§21–25), whose area model is src/fhe_sim/ppa.py with its CACTI calibration in calibration/cacti/; the deck build inlines them from results.md. fhe-sim --area prints the breakdown for any design.