Inside Ampere — 3rd-Gen Tensor Cores, 40 MB L2, MIG

Topics We'll Cover

• Ampere at a Glance — Two Dies, Two Foundries
• GA100 Die — Block Diagram
• GA102 Die — The Consumer Cousin
• The Ampere SM — Two Variants
• 3rd-Generation Tensor Cores — TF32, BF16, 2:4 Sparsity
• L2 Cache Jump — 6 MB → 40 MB
• Process — TSMC 7N vs Samsung 8N
• Voltage, Power, Form Factors
• Memory — HBM2e (Datacenter), GDDR6X (Consumer)
• NVLink 3.0 & NVSwitch 2 (HGX A100)
• Ampere's Innovations
• Interactive: Ampere SKU Picker

Ampere at a Glance — Two Dies, Two Foundries

Ampere launched in May 2020 as two physically distinct die families on two foundries. GA100 is the HPC/AI die at TSMC 7N. GA102 / GA104 / GA106 / GA107 are graphics dies at Samsung 8N (a custom 8 nm process). Same architecture name, very different silicon.

Datacenter (GA100) and consumer (GA10x) Ampere SMs are not the same SM. GA10x doubles FP32 cores per partition for graphics throughput; GA100 doubles FP64 cores instead. Read the next slides carefully.

GA100 Die — Block Diagram

GA102 Die — The Consumer Cousin

GA102 powers the RTX 3080 / 3090 / 3090 Ti, the RTX A6000 / A40 workstation cards, and the GA102-based Tesla A40. Process Samsung 8N, 28.3 B transistors on 628 mm².

Notable cuts vs GA100: FP64 lanes drop to 2/SM (1:64 of FP32); tensor cores stay 3rd-gen but fewer per SM at half the throughput; NVLink 3 reduced to a 4-link bridge on RTX 3090 / A6000, no NVSwitch.

The Ampere SM — Two Variants

3rd-Generation Tensor Cores — TF32, BF16, 2:4 Sparsity

Each Ampere SM has only 4 tensor cores (Volta had 8) but each is 4× the throughput — net 2× FP16 per SM. New formats are the headline:

• TF32 — 19-bit format with FP32 exponent (8 bits) and FP16 mantissa (10 bits). Drop-in replacement for FP32 matmul; 8× the throughput, no code change.
• BF16 — 8-bit exponent like FP32, 7-bit mantissa. Same dynamic range as FP32 with FP16 cost; the format that made stable mixed-precision training routine.
• 2:4 structured sparsity — in any 4-element block of weights, 2 must be zero; hardware skips the zeros, doubles throughput.

L2 Cache Jump — 6 MB → 40 MB

The L2 grew nearly 7× from Volta to GA100 — from 6 MB to 40 MB. Practical effect: working sets that previously thrashed HBM now fit on-die. For ResNet-50 inference at batch 1, weights fit in L2 entirely on A100. For LLM decoding the KV cache spills past it, but L2 buffering of recently-touched cache lines doubles effective bandwidth.

Ampere also added residency control: cudaStreamAttrAccessPolicyWindow lets you mark an address range as "persistent", giving its lines higher L2 retention priority. Used by cuDNN to keep convolution weights pinned.

GA10x consumer dies have only 6 MB L2 — not the headline jump. Consumer Ampere relies on bigger per-SM L1 (128 KB vs GA100's 192 KB but with consumer-tuned L1 hit rates) and high-BW GDDR6X.

Process — TSMC 7N vs Samsung 8N

NVIDIA split its Ampere supply between two foundries to manage capacity and cost. TSMC 7N for GA100 (premium-priced compute die, high-margin) and Samsung 8N for GA10x graphics dies (cheaper, larger volume). 8N is technically Samsung's 10 nm-class 8 nm LPP — not equivalent to TSMC N7. Density numbers:

The density gap (65.6 vs 45.1 M/mm²) explains why a 3090 needs more silicon than an A100 to look superficially similar — Samsung 8N is roughly an N12-class node by transistor density.

Voltage, Power, Form Factors

RTX 3090 Ti debuted the 16-pin 12VHPWR connector (later renamed 12V-2×6 after melting incidents on Ada). Voltage rails: GA100 has separate VDD, VDD-HBM, VDD-NVLink-IO, VDD-PLL; GA10x has VDD, VDDQ-GDDR6X (1.35 V), VDD-PLL.

Memory — HBM2e (Datacenter), GDDR6X (Consumer)

NVLink 3.0 & NVSwitch 2 (HGX A100)

NVLink 3.0 keeps per-link bandwidth the same as NVLink 2 (25 GB/s/dir, 50 GB/s bidirectional) but doubles the link count. A100 has 12 links → 300 GB/s/dir aggregate, 600 GB/s bidirectional. Signalling: 50 Gbps NRZ per lane, 4 lanes per link — reused the same 25 Gbps SerDes physical hardware as NVLink 2 but halved the lane count per link, with FEC (forward error correction) at the link layer to reach 50 Gbps reliably.

HGX A100 baseboard: 8× A100 SXM4 + 6× NVSwitch 2.0 chips (each NVSwitch 2.0 has 36 NVLink 3 ports, 1.8 TB/s aggregate). Every A100 talks to every other at full 600 GB/s. Cross-node uses 8× ConnectX-6 HCAs, 200 Gb/s HDR InfiniBand each.

Consumer Ampere (RTX 3090 / A6000 / A40) carries a 4-link NVLink bridge for two-card pairing: 112.5 GB/s bidirectional aggregate. RTX 3080 and below have no NVLink.

Ampere's Innovations

• 3rd-gen tensor cores: TF32, BF16, 2:4 sparsity. Made transformer training accessible at > 3× FP32 throughput with no code changes (TF32) and pushed BF16 mainstream.
• 40 MB L2 cache. Multi-MB L2 era begins. Working sets fit on-die.
• Async copy (cp.async). Hardware-asynchronous global-to-shared memory copy, frees registers from being staging buffers. The kernel-author win that made FlashAttention possible.
• MIG — Multi-Instance GPU. A100 can be partitioned into up to 7 hardware-isolated GPU instances with dedicated SMs, L2 slices, HBM regions, and memory bandwidth.
• NVLink 3.0 + NVSwitch 2. 600 GB/s per GPU, 8-GPU all-to-all in HGX A100; the multi-GPU TP era starts here.
• HBM2e at 2 TB/s. First production part to break the 2 TB/s memory bandwidth barrier (A100 80 GB).
• PCIe 4.0 host link. 32 GB/s/dir; first on a major NVIDIA part.
• Cooperative groups via tile size. CUDA 11 added tile-aware cooperative groups; underpins tile-based matmul libraries.
• RT 2nd gen + 7th-gen NVENC + 5th-gen NVDEC. Consumer-side wins (DLSS 2.0 maturing, AV1 decode debuts).
• Per-process address space ID (ASID) for MIG. Hardware support for the isolation guarantees MIG advertises.

Interactive: Ampere SKU Picker