MODERN SOC DESIGN · PRESENTATION 12
Dissipating 1000W+ from AI Silicon
| Failure Mechanism | Ea (eV) | AF @ +15 °C |
|---|---|---|
| Electromigration | 0.7 | 3.1× |
| NBTI | 0.5 | 2.2× |
| TDDB | 0.3–0.7 | 1.7–3.1× |
| Solder fatigue | — | Coffin-Manson |
Conduction · Convection · Radiation · Thermal resistance networks · RC models
| Accelerator | TDP (W) | Die Area | W/mm² |
|---|---|---|---|
| NVIDIA A100 | 400 | 826 mm² | 0.48 |
| NVIDIA H100 SXM | 700 | 814 mm² | 0.86 |
| NVIDIA B200 | 1000 | 2×~800 mm² (reticle, NV-HBI) | ~0.63 |
| AMD MI300X | 750 | 12 chiplets | ~0.65 |
| Intel Gaudi 3 | 900 (air) / 1200 (liq) | TSMC N5 OAM | — |
| Google TPU v5e | ~250 | ~400 mm² | ~0.63 |
| NVIDIA B300 | 1400 | 2×~800 mm² | ~0.88 |
| AMD MI355X | 1400 | CDNA4, TSMC N3 | — |
| NVIDIA Rubin VR200 (2026) | ~2300 | 2×reticle TSMC N3 | ~1.4 |
TIM fills microscopic air gaps between mating surfaces. Without TIM, contact resistance dominates: bare metal-to-metal leaves ~99% of area as air gaps (kair = 0.025 W/m·K).
| Material | k (W/m·K) | BLT (μm) | Rth (°C·mm²/W) |
|---|---|---|---|
| Thermal paste | 3–8 | 25–75 | 3–25 |
| Indium solder | 86 | 20–50 | 0.23–0.58 |
| Liquid metal (Ga alloy) | 40–73 | 10–30 | 0.14–0.75 |
| Graphite sheet | 10–15 (through) | 100–200 | 7–20 |
| STIM (solder TIM) | 50–86 | 15–25 | 0.17–0.50 |
Intel uses STIM (indium) on server CPUs; NVIDIA uses high-performance paste or STIM depending on SKU.
Heatsinks · Vapour chambers · Fan design · Server airflow management
≤200 W
Active heatsink
≤400 W
Heatpipe tower
≤700 W
Vapour chamber
A vapour chamber is a planar heat pipe: a sealed copper enclosure with an internal wick structure and working fluid (water). It provides near-isothermal heat spreading across the entire baseplate area.
| Spreading Solution | Rspread (°C/W) | Weight (g) |
|---|---|---|
| Solid copper (6 mm) | 0.08 | 580 |
| Copper + heat pipes | 0.04 | 320 |
| Vapour chamber (3 mm) | 0.015 | 180 |
Cold plates · Direct liquid cooling · Immersion · Coolant distribution
| Coolant | k (W/m·K) | cp (J/kg·K) | Notes |
|---|---|---|---|
| Water | 0.60 | 4186 | Best thermal; corrosion risk |
| Water-glycol 30% | 0.48 | 3600 | Freeze protection to −15 °C |
| PAO (dielectric) | 0.14 | 2100 | Safe for electronics; 3× worse |
| Novec 7100 | 0.07 | 1180 | Two-phase capable; expensive |
DLC delivers coolant directly to the chip package, bypassing heatsinks and airflow entirely. TSMC's direct-to-silicon approach achieves 0.055 °C/W at 2.6 kW — 3× better than the best air-cooled solutions.
| Cooling Method | PUE | kW/rack | Water Use |
|---|---|---|---|
| Traditional air (CRAC) | 1.5–2.0 | 10–25 | High (evap towers) |
| Hot aisle containment | 1.2–1.4 | 20–40 | Moderate |
| Direct liquid (DLC) | 1.05–1.15 | 40–100+ | Low (closed loop) |
| Single-phase immersion | 1.02–1.10 | 100–200 | Very low |
| Two-phase immersion | 1.01–1.06 | 200+ | Near zero |
Hotspot detection · Workload migration · Thermal throttling · 3D stacking challenges
Rack-level cooling · Hot/cold aisle · AI supercomputer cooling · ASHRAE guidelines
| Era | Rack Power | Cooling Method |
|---|---|---|
| 2015 — General compute | 5–10 kW | Raised floor air |
| 2018 — GPU training | 20–30 kW | In-row CRAC + containment |
| 2023 — H100 era | 40–70 kW | DLC + RDHx |
| 2025 — B200 / MI300X | 80–120 kW | Full DLC or immersion |
| 2026+ — Rubin / next-gen | 120–200 kW | Advanced liquid, microfluidic |
Presentation built with Reveal.js 4.6 · Saira Semi Condensed + Archivo + Share Tech Mono
Educational use. Data from public specifications and published papers.