Why I/O Bandwidth Is a First-Class Design Constraint
The I/O Bandwidth Problem An H100 GPU delivers 3.35 TB/s of HBM bandwidth internally, yet must communicate with the rest of the system over PCIe 5.0 ×16 at just ~128 GB/s — a 26× mismatch.
As ML clusters scale to thousands of accelerators, the interconnect fabric between chips, boards, and racks becomes the dominant performance and power constraint.
SerDes power ≈ 1–5 pJ/bit; at 10 Tbps aggregate this is tens of watts
Signal loss on PCB traces doubles every ~10 GHz — PAM4 is the industry response
Die-to-die interconnect (UCIe) targets <0.5 pJ/bit over µm-pitch bumps
Co-packaged optics eliminate long-reach electrical SerDes entirely for rack-scale
Every generation: 2–4× bandwidth, same pin count, lower power per bit
PCIe bandwidth doubles each generation; pin count stays fixed
A Serialiser/Deserialiser (SerDes) converts a wide parallel bus to a narrow high-speed serial stream and back, allowing a small number of pins to carry enormous aggregate bandwidth.
TX Path
PLL generates low-jitter serial clock
N:1 MUX serialises parallel data
FIR pre-emphasis boosts high-frequency content to pre-compensate channel loss
CML driver into 50 Ω differential pair
RX Path
CTLE: continuous-time linear equaliser (boost high freqs)
Key challenge: At 112 Gbps, one unit interval (UI) = 9 ps. Jitter budget for the entire link — TX, channel, and RX — must fit inside roughly 30% of that: ~2.7 ps rms.
05
NRZ vs PAM4 — The Modulation Shift
NRZ — Non-Return-to-Zero (2-level)
1 bit per symbol; symbol rate = bit rate
Simple slicer with single threshold at 0
Good noise margin; works well to ~28–56 Gbps
Baud rate equals bit rate; requires wide channel bandwidth
Used in: PCIe 4, 25G Ethernet, SATA, USB 3
PAM4 — Pulse Amplitude Modulation 4-level
2 bits per symbol; symbol rate = half the bit rate
4 amplitude levels: {−3, −1, +1, +3} (normalised)
Halves Nyquist frequency → lower channel loss at same bit rate
Eye height is ⅓ of NRZ → 9.5 dB worse SNR margin
Requires DSP-based equalisation; typically ADC + digital CDR
Used in: PCIe 5/6, 400G/800G Ethernet, HBM4, GDDR7
PAM4 Error Propagation
Each symbol carries 2 bits; a single symbol error can flip 1 or 2 bits
ISI (Inter-Symbol Interference): frequency-dependent loss smears symbols in time
Crosstalk: NEXT (near-end) and FEXT (far-end) from adjacent traces
Reflections: impedance discontinuities at vias, connectors, stubs
Jitter: deterministic (crosstalk-induced) + random (thermal / shot noise)
TX FIR Pre-emphasis
Finite Impulse Response filter applied at transmitter
Boosts high-frequency content before signal enters lossy channel
Main tap + 1–3 pre/post-cursor taps; coefficient magnitudes <1
De-emphasis variant: attenuates main tap relative to cursor taps
PCIe 5: 11 TX FIR coefficients, 3 pre-cursors, 10 post-cursors
CTLE — Continuous-Time Linear Equaliser
Analogue high-pass filter at RX input
Boosts high frequencies to invert channel roll-off
Implemented with source-degenerated differential pair
Gain peaking: +6 to +20 dB at Nyquist; limited by noise amplification
Programmable via capacitor bank / resistor ladder
DFE — Decision-Feedback Equaliser
Uses decided past symbols to cancel post-cursor ISI
No noise amplification (unlike linear equalisers)
H1 tap critical: must complete in one UI (speed-limiting)
Unrolled (speculative) DFE: solves H1 timing with 2× slicers
4–8 taps typical for 56 G NRZ; 8–16 taps for 112 G PAM4
Adaptation
Coefficients adapt continuously using LMS (Least Mean Squares) or
MMSE minimising RX eye closure. Convergence: ~10⁵–10⁶ symbols.
PCIe link training runs adaptation sequences at startup.
08
Clock & Data Recovery (CDR)
The receiver has no reference clock aligned to the incoming data — it must extract timing directly from data transitions. CDR is the circuit that does this.
Phase-Locked Loop (PLL) CDR
Phase detector compares data edges vs VCO phase
Loop filter averages error signal
VCO tracks frequency + phase of incoming data
High bandwidth PLL: fast lock, higher jitter peaking
Used in: NRZ links, SATA, USB, older Ethernet
Phase Interpolator (PI) CDR
Fixed RX PLL generates reference phases (e.g. 8)
Digital loop selects/blends phase for sampling
Higher jitter tolerance bandwidth flexibility
Dominant in 56G+ SerDes, PCIe 4/5/6
Digital phase controller: binary/bang-bang or Mueller-Müller
Bang-Bang (BB) Phase Detector
Samples data at two phases: "early" and "late" relative to eye centre
1-bit decision: phase error is always ±1 LSB (non-linear)
Simple, fast, but causes jitter at steady state (limit cycling)
Loop bandwidth set by proportional + integral filter coefficients
Digital CDR (ADC-based, PAM4)
PAM4 at 56G+ often uses a flash ADC (4–6 bits) followed by a fully digital CDR and DFE running at fractional rate. Enables aggressive EQ and adaptation but consumes significant area and power.
9 ps
1 UI at 112 Gbps — total jitter budget
09
PCIe 5 & 6
Protocol stack · Link training · FLIT mode · Bandwidth scaling
10
PCIe Protocol Stack
TLP — Transaction Layer Packet
Header (3–4 DW): type, address, length, requester/completer ID, tag
Credit-based: receiver advertises header + data credits per VC
Transmitter counts credits before sending; no retransmit needed
3 traffic classes × 2 VC queues; QoS isolation
DLLP Ack/Nak: link-level retry on LCRC failure
PCIe 6.0 — FLIT Mode
Fixed-size 256-byte FLIT (Flow Control Unit) replaces variable TLP framing
Enables FEC (Reed-Solomon + CRC-6) → BER <10⁻¹⁸ from raw 10⁻⁴ PAM4
Retransmit removed: FLIT errors recovered by FEC → lower latency
64 GT/s per lane × 16 lanes = 128 GB/s per direction (256 GB/s bidirectional)
Generation
Encoding
GT/s
BW ×16
Modulation
PCIe 3
128b/130b
8
16 GB/s
NRZ
PCIe 4
128b/130b
16
32 GB/s
NRZ
PCIe 5
128b/130b
32
64 GB/s
NRZ
PCIe 6
FLIT/256B
64
128 GB/s
PAM4
PCIe 7
FLIT/256B
128
256 GB/s (×16, bidir)
PAM4
PCIe 7.0 finalised by PCI-SIG in June 2025; PCIe 8.0 pathfinding underway.
11
PCIe Link Training & State Machine (LTSSM)
Before data flows, each PCIe link must negotiate parameters and equalise the channel. The Link Training and Status State Machine (LTSSM) manages this sequence.
Configuration: lane width + link number negotiated
Recovery: re-equalises on link errors or speed change
L0: normal active operation
L0s: electrical idle; fast exit (~1 µs)
L1: deeper power-down; slow exit (~10–100 µs)
L2/L3: link off; power removed
Equalization Phases (PCIe 3+)
Phase 1: TX uses default preset (P0–P10 for PCIe 4/5)
Phase 2: receiver requests TX preset changes via TS2; optimises TX FIR
Phase 3: receiver adapts CTLE & DFE to current TX setting
Phase 4: (PCIe 6) additional FLIT-mode training with FEC check
Full 3-phase EQ: ~10–50 ms; critical for boot time & hot-plug latency
ASPM — Active-State Power Management
Hardware autonomously transitions idle links to L0s/L1 to save power. L1 substates (L1.1, L1.2) use CLKREQ# handshake to gate reference clocks. Critical for edge AI SoCs: a PCIe 5 ×16 link idles at >2 W.
Why a Die-to-Die Standard? Chiplet architectures (AMD, Intel, Apple, NVIDIA) use proprietary die-to-die interconnects. UCIe (announced 2022, now v3.0 with 64 GT/s) provides an open standard so chiplets from different vendors can interoperate — the "PCIe of the package."
UCIe Physical Layer
Standard package: 55 µm bump pitch; up to 16 GT/s NRZ
Advanced package: 25 µm bump pitch (CoWoS/SoIC); up to 64 GT/s (UCIe 3.0)
64-bit data bus per module; sideband channel
LPCLK forwarded; no CDR needed (common clock)
FDI (flexible die interface) optional FEC
Bandwidth Density
Standard: ~1 Tb/s/mm edge bandwidth
Advanced: ~10 Tb/s/mm (2 TB/s per mm)
PCIe 5 off-package: ~2 GB/s/mm²
UCIe advanced: ~10–40 GB/s/mm²
UCIe Protocol Stack
Protocol layer: PCIe TLP or CXL.io/cache/mem — runs unchanged over UCIe
The Pluggable Optical Module Problem At 400G+, pluggable QSFP modules sit at the front panel — 1–2 metres of electrical trace between switch ASIC and optics. At 112 Gbps PAM4 per lane this trace dissipates ~5–10 W per module just in SerDes and retimer power. CPO moves the photonic engine inside the package, replacing long electrical traces with micron-scale optical waveguides.
Silicon Photonics PIC
Mach-Zehnder modulators (MZM) or micro-ring modulators (MRM)
Germanium photodetectors (25+ GHz BW)
Spot-size converters for fibre coupling
Fabricated on 200–300 mm Si wafer (GlobalFoundries, TSMC, imec)
EIC + PIC Integration
Electronic IC (laser driver, TIA) flip-chipped onto PIC
Bump pitch 50–100 µm; µm-scale electrical trace to modulator
ML SoC: co-packaged photonic die for scale-out interconnect
224G
next-gen lane rate PAM4 in R&D (2025+)
<2 pJ
target energy/bit for 1.6T-era SerDes
22
Summary
SerDes Architecture
TX PLL + FIR serialiser; RX CTLE + DFE + CDR deserialiser. 112G PAM4 state-of-art with ADC-based digital DSP. Jitter budget <3 ps rms at 1 UI = 9 ps.
NRZ vs PAM4
PAM4 halves baud rate for same bit rate; reduces channel bandwidth requirement. 9.5 dB SNR penalty demands aggressive EQ and mandatory FEC (RS KP4).
UCIe
Open die-to-die standard. Advanced package: 25 µm pitch, 10+ TB/s/mm bandwidth density — 20× denser than PCIe off-package. PCIe/CXL protocol tunnelled unchanged.
The Trajectory:
Electrical SerDes pushes toward 224G PAM4; beyond that, insertion loss mandates optical solutions.
Co-packaged optics will dominate rack-scale AI interconnect by 2027, reducing the optical module power by 4–8× vs pluggables and enabling per-chip I/O bandwidths exceeding 10 Tbps.
Up Next in the Series
05 · AI / ML Accelerator Architectures
06 · SoCs for ML Inference
07 · Security in SoC Design
References
PCI-SIG: PCIe 5.0 / 6.0 / 7.0 (2025) Base Specifications