ML Accelerator SoC
Power Delivery

Why It Is Getting Harder

• Lower Vdd — sub-0.8 V rails shrink absolute noise margin
• Higher current density — >2 A/mm² in modern logic tiles
• Bursty workloads — GEMM→all-reduce transitions create massive di/dt
• Larger die / multi-die — reticle-limit dies and chiplet assemblies lengthen PDN
• Thermal co-stress — hot spots raise local resistance, worsening IR drop

Consequences of Poor Power Delivery

• Timing failures — voltage droop slows gates, violates setup constraints
• Frequency guardband — 10–15% clock reduction to tolerate worst-case droop
• Electromigration — J > J_crit accelerates void formation (Black’s law)
• Reliability — repeated thermal cycling fatigues solder bumps/pillars
• Power waste — voltage guardbands × current = wasted Watts

Impedance Budget by Stage

Typical Numbers — 700W AI GPU

Target Impedance Method

• Flat impedance — maintain Z < Z_target from DC to ~1 GHz
• Anti-resonance peaks — occur at transitions between cap stages; must be damped below target
• Series resonance of each cap stage creates a “valley” in Z(f) where that stage dominates
• Parallel resonance between adjacent stages creates dangerous peaks
• Analysis via S-parameter measurements or 3D EM extraction tools (Ansys SIwave, Cadence Sigrity)

Frequency Regimes

Anti-Resonance Mitigation

• Overlap ESR-damped capacitor values to smooth transitions
• Use controlled-ESR capacitors to flatten peaks
• Distribute package caps spatially to reduce mutual inductance
• eDTC on-die capacitance pushes mid-frequency Z below target

Static IR Drop

• V_drop = I × R_path from pad to transistor
• Dominated by metal stack sheet resistance (Cu at 5nm: R_sheet ~50–100 mΩ/sq for M1)
• Worst at die center — longest path from peripheral power bumps
• Typical budget: 20–40 mV static drop (3–5% of Vdd)
• Mitigation: thicker upper metals (M12–M15 at 3–5 µm), denser bump grid, BSPDN

Dynamic IR Drop

• V_drop = L × di/dt + I × R — inductive component dominates during transients
• Triggered by simultaneous switching of large logic blocks
• Can exceed static drop by 2–5× during worst-case events
• Duration: typically 1–50 ns droop events
• Mitigation: decoupling, staggered clock enables, power-aware placement

Electromigration & Black's Equation

• J — current density (A/cm²); n ≈ 1–2 for Cu
• E_a — activation energy (~0.7–0.9 eV for Cu dual-damascene)
• T — temperature; 10°C rise → ~2× EM rate acceleration
• J_max for Cu at 105°C: ~1–2 MA/cm² (varies by process / metal level)
• Power grid wires must be sized for EM limits, not just IR drop

Current Density Maps

• Generated via vectorless or VCD-based dynamic analysis
• Hot spots correlate with high-activity macro blocks (tensor cores, SRAM banks)
• Tools: Cadence Voltus, Synopsys RedHawk, Ansys Totem
• Signoff criteria: max dynamic droop < 8–10% Vdd, EM margin >10 years at 105°C

Multi-Phase Buck Converter

• Topology — N interleaved buck phases (typically 12–20 phases for >600A)
• Switching freq — 400–800 kHz per phase; effective ripple at N×f_sw
• Current sharing — DCR sensing or hall-effect; <5% imbalance target
• Efficiency — 90–93% at full load for 12V→0.75V conversion
• Transient response — droop during 0→full-load step: V_droop = L_out × di/dt
• Output filter — L: 50–150 nH/phase, C_out: 10–30 mF total

48V Direct Conversion

• OCP Open Rack V3: 48V bus eliminates 12V intermediate stage
• 4:1 ratio — 48V → ~12V via switched-cap, then multi-phase buck to Vcore
• Or direct 48V→0.75V using high-ratio topologies (Vicor, Infineon)
• Reduces I²R losses in PCB (4× lower current at 48V vs 12V)
• NVIDIA HGX B200: 48 V input, 1000 W per GPU (GB200 Grace+Blackwell superchip ~2.7 kW; B300 1400 W)

DrMOS & Smart Power Stages

• DrMOS: Driver + high-side + low-side FETs in single package (8×8 mm)
• Infineon TDA21490: 90A continuous, R_DS(on) < 0.5 mΩ
• Renesas ISL99390: 90A, integrated temperature sensing
• GaN power stages — higher f_sw (2–4 MHz), smaller inductors, faster transient
• EPC2302: GaN FET, 1.3 mΩ, enables >2 MHz switching

VRM Spec Example — AI GPU Card

Intel Fully Integrated Voltage Regulator (FIVR)

• First deployed in Haswell (2013); removed after Broadwell in client; partially restored in Ice Lake/Sapphire Rapids server; modern Intel client (Meteor/Arrow/Lunar Lake) uses package VRM + on-die DLVR (digital linear VR)
• Topology — on-die multi-phase buck with thin-film inductors on package
• Switching frequency: 60–140 MHz (100× higher than board VRM)
• Enables per-core DVFS with <100 ns voltage transition
• Inductor: air-core thin-film, ~1–2 nH, embedded in package substrate
• Efficiency: 85–90% (trades ~3% vs board VRM for faster response)

Advantages Over Board VRM

• Faster transient response — 100 ns vs 10–50 µs for board VRM
• Per-domain regulation — separate rails for each core, GPU, IO, etc.
• Reduced guardband — 30–50 mV savings enables higher frequency or lower power
• Fewer board components — simpler motherboard, smaller form factor
• Dynamic power savings — rapid DVFS during idle/burst cycles

PIVR in AI Accelerators

• Intel Ponte Vecchio — FIVR with per-tile regulation across 47 chiplets
• Intel Gaudi 3 — FIVR for independent HBM and compute voltage domains
• Package inductors: 2.5D embedded thin-film on EMIB interposer
• Trade-off: FIVR consumes ~2–4% die area for power FETs + control

On-Die LDO (Low Dropout Regulator)

• Topology — PMOS pass transistor + error amplifier, fully in standard CMOS
• Dropout — 50–100 mV minimum (V_in − V_out)
• Efficiency — η = V_out / V_in ≈ 85–93% (limited by dropout)
• Bandwidth — loop BW up to 100–500 MHz — excellent transient response
• Area — ~0.01–0.05 mm² per LDO instance, distributed across die
• Use case — per-core, per-SRAM-bank, or per-tensor-unit voltage islands

Digital LDO

• Array of parallel PMOS switches with binary-weighted widths
• Digital comparator + shift register replaces analog error amp
• Advantage — fully synthesizable, scales with process node
• Disadvantage — limit-cycle oscillation, slower response than analog LDO
• Hybrid designs combine coarse digital + fine analog paths

On-Die Switched-Cap IVR

• Charge-pump topology: 2:1 or 3:2 ratio conversion using on-die MOS caps
• Efficiency — >90% at fixed rational conversion ratios
• No inductors required — fully CMOS compatible
• Flying capacitors: MOS-cap or MIM-cap in BEOL stack
• Limitation — fixed ratio; cascaded with LDO for fine regulation
• Apple M-series: switched-cap + LDO for per-cluster voltage islands

Trade-offs: Board vs Package vs Die Regulation

How BSPDN Works

• Conventional (frontside) — power & signal share the same BEOL metal stack
• BSPDN — power rails delivered through the backside of the wafer via nano-TSVs
• Wafer is thinned to ~1–3 µm, backside metallization added (2–4 metal layers)
• Nano-TSVs (~50–200 nm diameter) connect backside power to frontside transistors
• Signal routing uses full BEOL stack — no power rails consuming M1/M2 tracks

Intel PowerVia (Intel 20A / 18A)

• First production BSPDN: Intel 18A (2025) — Intel 20A was cancelled as a HVM node; 18A ships in Panther Lake (client) and Clearwater Forest (Xeon 6+)
• Backside power with RibbonFET (gate-all-around) transistors
• 30–50% IR drop reduction vs frontside at iso-density
• 6–10% frequency uplift from reduced voltage guardband
• Cell height reduction: enables 6T SRAM at higher density

Benefits of BSPDN

• IR drop — 30–50% reduction; shorter, wider power paths to transistors
• Signal routing — 15–25% more tracks available (no VDD/VSS on M1/M2)
• Cell density — tighter standard cell pitch; potential 10–20% area reduction
• Decoupling — backside metal planes act as distributed capacitance
• Thermal — direct backside metal contact to heat sink / cold plate

TSMC Backside Power (N2P, A16)

What is eDTC?

• Deep trenches etched into the silicon substrate, lined with high-k dielectric and filled with conductor
• Capacitance density: 250–400 nF/mm² (vs ~20 nF/mm² for MOS-cap filler)
• 10–20× higher capacitance density than conventional on-die MOS decaps
• Trenches are ~5–10 µm deep, ~100–200 nm wide, etched into Si substrate
• Process: adapted from DRAM deep trench technology

Impact on PDN Impedance

• 2–5× reduction in on-die PDN impedance at 200 MHz–2 GHz
• Pushes anti-resonance peaks below target impedance
• Enables flatter Z(f) profile across critical frequency band
• Reduces need for package-level decoupling caps (cost saving)
• Particularly impactful for large-die ML accelerators with high di/dt

TSMC eDTC Technology Roadmap

TSV Power Delivery Basics

• TSV diameter — 5–10 µm (standard), 1–3 µm (fine-pitch for 3D)
• Resistance per TSV — ~10–50 mΩ (depending on diameter/depth)
• Current per TSV — 10–50 mA per via at EM limits
• Pitch — 30–100 µm; dense arrays needed for high-current delivery
• For 500A total: need ~10,000–50,000 power TSVs

2.5D Interposer PDN (CoWoS)

• TSMC CoWoS-S: silicon interposer with TSVs connecting package to chiplets
• Interposer TSVs: 10 µm diameter, 100 µm depth, ~25 mΩ each
• Power planes in interposer RDL layers (3–4 metal layers)
• Decoupling capacitors can be placed on interposer (MIM caps)
• Total power TSV count for HBM+GPU assembly: 50,000+

3D Stacking Power Challenges

• Thermal stacking — top die heated by bottom die; T_junction rises → higher R → worse IR drop
• TSV area overhead — 5–15% of die area consumed by TSV keep-out zones
• Reliability — Cu TSV pumping, thermo-mechanical stress, Cu/oxide delamination
• IR drop compounds — power must traverse multiple die layers in series
• AMD 3D V-Cache: ~100k TSVs for power + signal between CCD and cache die

Thermal-Power Co-Design

NVIDIA Blackwell Power Architecture

• B200 module — 2 GPU dies on CoWoS-L substrate + 8 HBM3E stacks
• 48V input — 48V→~6V (switched-cap IBC) → 0.75V (multi-phase buck)
• VRM — 20+ phase digital multi-phase controller per die
• Power connector — 16-pin 12VHPWR (or 48V direct via backplane)
• PCB — 16+ layer, 2oz+ Cu, dedicated power planes per voltage rail
• Package — CoWoS-L with >60,000 power bumps per die

48V to Point-of-Load Path

The di/dt Problem in ML Training

• GEMM phase — all tensor cores active, ~100% utilization, peak current draw
• Communication phase — NVLink/ICI all-reduce; tensor cores idle, current drops 40–60%
• Transition — 500–1000A swing in <1 µs → massive Ldi/dt voltage droop
• Droop magnitude: 50–100 mV for worst-case transitions
• Occurs predictably at layer boundaries in transformer training
• Frequency: every 100–500 µs during typical LLM training iterations

Hardware Mitigation

• On-die decoupling — eDTC + MOS caps provide charge reservoir for <50 ns events
• Staggered clock gating — spread wake-up across 10–50 ns window
• Current-step limiters — on-die circuits that limit max di/dt by throttling clock enables
• Adaptive clocking — drooping clock tracks supply voltage, maintaining timing margins
• PIVR/FIVR — fast VRM response (<1 µs) catches medium-duration droops

Firmware & Software Mitigation

• Workload scheduling — overlap compute and communication to smooth transitions
• DVFS pre-positioning — lower voltage before known idle → reduce droop magnitude
• Droop predictor — ML model predicting upcoming di/dt events, pre-activating mitigations
• Power throttling — firmware caps max current ramp rate during phase transitions
• NVIDIA GPU firmware: dynamic power balancing across SMs to limit aggregate di/dt

Signoff Tool Landscape

Analysis Flow

1. Extract die power grid (RLC network, 10M+ nodes)
2. Map switching activity from VCD or vectorless estimation
3. Combine with package/PCB model (S-parameter or lumped RLC)
4. Run transient simulation over representative workload window
5. Report worst-case droop map, EM hotspots, voltage histogram

Workload-Driven Power Integrity

• VCD-based — captures exact switching; 10–100 TB of data for full-chip AI accelerator
• Vectorless — statistical estimation; faster but less accurate
• Hybrid — VCD for critical blocks (tensor cores), vectorless for rest
• ML workload models — abstract current waveforms for GEMM, conv, attention phases
• Signoff corner: worst-case voltage at slowest process, highest T, max current

Modern Signoff Criteria

Chiplet PDN Challenges

• Multiple voltage domains — compute, IO, HBM PHY each need distinct Vdd
• Cross-chiplet transients — one chiplet's di/dt couples through shared interposer PDN
• Interposer IR drop — silicon interposer adds 5–20 mΩ series resistance per path
• Power bump density — limited by µ-bump pitch (40–55 µm for CoWoS)
• Independent DVFS — requires per-chiplet VRM or IVR with isolation

Per-Chiplet Voltage Regulation

• Option A — board VRM per chiplet (separate power domain from PCB)
• Option B — PIVR per chiplet (Intel Ponte Vecchio approach)
• Option C — shared VRM + on-die LDO per chiplet (AMD MI300X)
• Trade-off: board VRM is efficient but slow; on-die LDO is fast but wastes ~5–10%
• Emerging: GaN-based package-integrated regulators for per-chiplet fast regulation

Power Gating Coordination

• Inrush current — enabling a power-gated chiplet draws 50–200A in 100 ns
• Sequencing — stagger chiplet wake-up to limit aggregate di/dt
• Retention — state retention power gating keeps SRAM live at lower voltage
• Protocol — chiplet power manager communicates with global power controller via sideband
• AMD MI300X: 8 XCD + 4 IOD chiplets with coordinated power management

Interposer PDN Design

Key Papers & Publications

• M. Swaminathan & A.E. Engin, “Power Integrity Modeling and Design for Semiconductors and Systems”, Prentice Hall, 2007
• L. Smith, R. Anderson, T. Roy, “Power Distribution System Design Methodology and Capacitor Selection for Modern CMOS Technology”, IEEE Trans. AP, 1999
• N. Sturcken et al., “A Switched-Inductor Integrated Voltage Regulator with Nonlinear Feedback”, IEEE JSSC, 2012
• E. Burton et al., “FIVR — Fully Integrated Voltage Regulators on 4th Gen Intel Core SoCs”, IEEE APEC, 2014
• TSMC, “eDTC: Deep Trench Capacitor for Advanced PDN”, IEDM 2022

Industry Standards

• OCP Open Rack V3 — 48V rack power distribution specification
• PCI-SIG CEM 5.0 — 12VHPWR connector (600W) specification
• JEDEC JEP-148 — Electromigration reliability qualification guidelines
• IEEE 1801 (UPF) — Unified Power Format for power intent specification
• JEDEC JESD51 — Thermal measurement standards for semiconductor packages

Conference Venues

• IEEE ECTC — Electronic Components and Technology Conference (packaging & PDN)
• IEEE IEDM — International Electron Devices Meeting (eDTC, BSPDN process)
• IEEE ISSCC — International Solid-State Circuits Conference (IVR, LDO design)
• IEEE APEC — Applied Power Electronics Conference (VRM, power conversion)
• DesignCon — Signal & power integrity, PDN measurement techniques
• Hot Chips — AI accelerator architecture & power delivery overviews

EDA & Simulation Resources

• Cadence Voltus / Sigrity — die + package power integrity co-simulation
• Synopsys RedHawk / PrimeRail — EM/IR signoff, adaptive voltage analysis
• Ansys SIwave / HFSS — 3D EM extraction for package PDN
• Keysight ADS / PathWave — board-level PDN simulation & measurement
• Open-source: OpenROAD — IR drop analysis in open-source EDA flow