One real channel, characterised and then equalised all the way to a link budget — and an honest answer about which fix is worth the money when it still does not close.
A transmitter launches a clean rectangular symbol. A receiver, twenty-nine inches later, sees something that barely resembles it. Four mechanisms are responsible, and each is fixed by a different block — which is why it pays to keep them apart.
Copper loss rises as $\sqrt{f}$ as current crowds into a thinner skin; dielectric loss rises roughly as $f$. A square pulse is a wide band, so the channel does not merely shrink it — it smears it across many symbols. That is intersymbol interference, and it dominates everything here.
Every discontinuity sends part of the wave back, and part of that returns again. Energy arriving late is ISI with a longer memory and a less predictable shape.
Unlike the first two, it is not correlated with your own data, so it behaves as noise rather than as ISI — and no equaliser can subtract it.
Moves the sampling instant rather than the level, converting to amplitude error in proportion to the slope of the waveform where you sample it.
Because the chassis, the connector and the laminate are chosen long before the SerDes is, and none of them get cheaper with each doubling of data rate. Equalisation is the part of the problem that lives on silicon, which is the only part that does. That is why every rate increase for twenty years has been paid for with more equalisation rather than with better copper — and also why slide 10, where the board turns out to be the cheaper fix, is worth reading carefully.
XAUI in 2002 ran 3.125 GBd over a channel with perhaps 6 dB of loss at Nyquist, and a fixed two-tap transmit de-emphasis — a switchable current in the output driver — was enough. Nothing about it adapted; a board designer picked a setting from a table.
The step that made modern links possible was 10GBASE-KR in 2007, which standardised three things at once: a three-tap transmit FFE, a receive DFE, and a training protocol by which the receiver tells the transmitter which way to move its taps. Until then equalisation was open-loop and set by hand. After it, a link negotiated its own settings at start-up and 20 dB channels became routine.
CEI-28G around 2011 added the CTLE as a standard receive block and pushed decision feedback to a dozen taps. That is the generation this deck's worked example belongs to, and the last in which a designer can follow every block by hand. What came next changed the character of the problem: 400GbE and CEI-56G moved to PAM4 and made forward error correction mandatory, because the 9.5 dB PAM4 gives away (slide 11) is simply not recoverable by equalisation. 802.3ck and CEI-112G then replaced the analogue slicer with an ADC, moving equalisation into the digital domain where long filters and sequence estimation become affordable.
Every generation bought its rate increase with more equalisation on silicon, not with better copper. The structure of the problem has not changed since 2007: a channel you cannot alter, a pulse response that describes it completely, filters whose taps solve a quadratic minimisation, and a budget that decides whether it works.
A conventional two-card backplane topology: transmit package, six inches of line card, a connector, eighteen inches of backplane, a second connector, four inches on the receiving card, receive package. 28.8 inches of 100 Ω differential stripline on Dk 3.7 laminate at 163 ps per inch — a flight time of 4.7 ns, or 131 unit intervals. There are always more than a hundred symbols in flight at once, which is worth remembering when reasoning about what a reflection does.
Three things to read off the S-parameters before designing anything. Insertion loss at Nyquist is the headline difficulty. Return-loss ripple is the signature of energy bouncing between the two connectors; because the round trip is many unit intervals, those echoes land as isolated ISI terms far from the cursor, exactly where a five-tap DFE cannot reach them. And mode conversion, the $\mathbf{S}_{cd}$ block of the companion deck, matters here for a reason specific to equalisation: converted energy is energy the receiver never sees, so it shrinks the cursor without ever appearing in $S_{dd21}$.
Which is why characterisation is done on a multiport fixture. Near-end and far-end crosstalk from the neighbouring lanes are aggregated into one figure of merit, the integrated crosstalk noise, and it enters the budget on slide 09 as 3.0 mV rms. On this channel it turns out to be the largest single term.
A plated through-hole that carries the signal only part of the way down leaves the rest of the barrel hanging off the path as an open-circuited stub. An open stub is a quarter-wave short: at the frequency where its length is a quarter wavelength it presents a near short-circuit across the line, and the channel develops a deep notch.
$$f_{\text{notch}} = \frac{1}{4\,\tau_{pd}\,\ell}, \qquad \tau_{pd} = 163\ \text{ps/inch}.$$
Drag the length. Watch where the notch lands relative to Nyquist.
On the modelled channel the as-fabricated 110 mil stub adds about 25 dB at Nyquist, taking $|S_{dd21}|$ from −33.3 dB to −58.2 dB. No amount of equalisation recovers a notch: the information at those frequencies is gone, not merely attenuated. Back-drilling is not an optimisation here, it is the difference between a link and a brick.
Insertion loss tells you how much of a sine wave survives. It does not directly tell you what a receiver sees, because a receiver does not sample sine waves — it samples one symbol at a time, in the presence of all the others. The object that answers that question is the pulse response: the waveform produced by sending a single one-UI symbol and nothing else. Inverse-transform $S_{dd21}$, convolve with a one-UI rectangle, and there it is.
Sample that waveform once per unit interval at the phase that maximises the peak, and the resulting sequence is the whole story:
The largest sample — the part of the symbol that arrives when it is supposed to. Here 0.199 of the transmitted amplitude.
Energy that arrived early, landing on symbols not yet decided. The DFE can never touch these. First one: 0.117.
Energy still arriving one, two, ten symbols later. First one: 0.145, and the tail runs for dozens of UI.
The cursor is 0.199 and the sum of the absolute values of every other tap is 0.636 — more than three times larger. The worst-case eye opening is twice the difference, which comes out at −0.87. A negative eye opening is not a small eye: it means that for some patterns of neighbouring data the receiver sees the wrong sign entirely. Nothing about this link works without equalisation.
Four blocks, each good at something the others are not, and each with a price.
| Block | Removes | Cannot remove | Costs |
|---|---|---|---|
| CTLE analogue, in the front end | Broadband slope, very cheaply | Anything with structure — reflections, isolated echoes | Cursor amplitude. It attenuates low frequencies rather than amplifying high ones |
| Transmit FFE FIR before the channel | Pre-cursor and near post-cursor ISI, before crosstalk is added | Anything it cannot see; it has no knowledge of the receiver's noise | Transmit swing, through the peak-power constraint $\sum|c_i| \le 1$ |
| Receive FFE FIR after the channel | Pre-cursor ISI and long tails, with many taps available | Nothing in principle — but every tap costs noise | Noise enhancement: 0.75 dB for the nine-tap MMSE design here |
| DFE feedback on decisions | Post-cursor ISI exactly, with no noise penalty at all | Pre-cursor ISI, ever. It acts only on symbols already decided | Error propagation, and a hard timing path on the first tap |
A DFE subtracts the known contribution of symbols it has already decided, so it cannot touch pre-cursor ISI — that comes from symbols it has not decided yet. A receive FFE can, at the cost of noise enhancement. A transmit FFE can, at the cost of swing. Splitting the work so the transmitter handles pre-cursors and the feedback path handles post-cursors is not arbitrary: it puts each impairment where the cheapest remedy is.
The real pulse response of the modelled channel, equalised in the browser. Choose a CTLE setting, switch the transmit FFE in or out, set the number of feedback taps, and adjust the noise the receiver has to live with. Everything below is computed from the sampled pulse response, not looked up.
Three things are worth trying. Turn the CTLE down and watch the cursor rise while the ISI rises faster. Turn the transmit FFE off and watch the pre-cursors reappear where no DFE can reach them. And push the DFE past about eight taps and watch the returns flatten out — the tail is long, but by then it is small compared with the crosstalk.
Putting the blocks in series and tracking the pulse response through them gives the table below. The cursor falls monotonically, which is the part that surprises people new to this: equalisation is not about making the signal bigger, but about making the interference smaller faster than the signal shrinks.
| Stage | Cursor | Σ|ISI| | Worst-case eye | Verdict |
|---|---|---|---|---|
| Channel only | 0.199 | 0.636 | −0.87 | Shut, and inverted for some patterns |
| + CTLE at −12 dB | 0.070 | 0.139 | −0.14 | Still shut |
| + transmit FFE | 0.032 | 0.036 | −0.007 | On the edge |
| + DFE, five taps | 1.000 (normalised) | 0.349 | 1.30 | Open, but not by much |
$[\,0.040,\ -0.340,\ 0.620\,]$ — two pre-taps, zero-forcing the first two pre-cursors, normalised so $\sum|c_i| = 1$. The main tap at 0.62 is the de-emphasis penalty made visible.
$[\,0.301,\ -0.368,\ -0.195,\ -0.032,\ 0.005\,]$. A second tap of −0.37 is large enough that error propagation is a real risk, which is one reason FEC and decision feedback are usually specified together.
The feedback path must settle within one unit interval — 35.7 ps here — from slicer output back to slicer input. That puts the first tap on the critical path of the whole receiver, which is why it is usually unrolled: both possible answers are computed speculatively in parallel and the decision selects between them after the fact.
An open eye is necessary but not sufficient. What decides the error rate is the ratio of the eye opening to everything that moves the received level around, and those contributions must be referred to a common point. Referring them to the receiver input, where the cursor is 12.9 mV out of an 800 mV transmitted swing:
| Contribution | Value | Where it comes from |
|---|---|---|
| Crosstalk | 1.58 mV rms | 3.0 mV of integrated crosstalk noise from neighbouring lanes, shaped by the CTLE |
| Residual ISI | 1.12 mV rms | What the equalisers left behind, treated statistically rather than worst-case |
| Receiver noise | 0.95 mV rms | 1.8 mV of front-end noise, shaped by the same filter (−5.6 dB over the noise bandwidth) |
| Jitter | 0.10 mV rms | 350 fs rms of random jitter, converted through the slope at the sampling instant |
| Total | 2.15 mV rms | Root-sum-square |
Signal 12.9 mV over noise 2.15 mV gives $Q = 6.01$, a bit error rate near $10^{-9}$. The conventional target for an uncorrected link is $10^{-12}$, which needs $Q = 7.03$. This link is 1.36 dB short.
Crosstalk is the largest term, residual ISI is second and receiver noise third. That ordering matters: it says the link is short of margin mostly because of its neighbours and only partly because of what the equalisers left behind. Removing every last scrap of residual ISI would take the total from 2.15 mV to about 1.8 mV — worth roughly four tenths of a decibel, nothing like the shortfall. The next slide checks that arithmetic against every alternative.
Seven candidate remedies, each re-run through the whole chain from the channel model outwards, so the margin figures are computed rather than estimated.
| What you change | Loss at Nyquist | Q | Margin to $10^{-12}$ |
|---|---|---|---|
| As built | −33.3 dB | 6.01 | −1.36 dB |
| Halve the crosstalk (connector + pair pitch) | −33.3 dB | 7.77 | +0.87 dB |
| Low-loss laminate (Df 0.005 → 0.003) | −28.0 dB | 8.98 | +2.12 dB |
| Shorten the backplane 18″ → 13″ | −29.3 dB | 8.52 | +1.67 dB |
| Ten DFE taps instead of five | −33.3 dB | 6.20 | −1.10 dB |
| Add a 9-tap receive FFE (the ADC-DSP answer) | −33.3 dB | 6.34 | −0.90 dB |
| Low-loss laminate + halved crosstalk | −28.0 dB | 11.38 | +4.18 dB |
A lower-loss laminate lifts the cursor by reducing the slope the equalisers have to undo. Five inches off the backplane does much the same, often more cheaply. Halving the crosstalk attacks the largest single term directly. Any one of the three closes the link.
Doubling the DFE taps buys 0.26 dB. Adding a nine-tap receive FFE — the block that defines a modern ADC-DSP receiver, and the obvious thing to reach for — buys 0.46 dB, and still leaves the link short.
The block that fixed the last problem is rarely the block that fixes the next one. A link short of margin is short for a reason; the budget names the reason; and the instinct to reach for more equalisation is the instinct to solve the problem you solved last time. On a channel dominated by ISI rather than crosstalk this table would look entirely different — which is exactly why you build the budget first.
The obvious way to double throughput without doubling the Nyquist frequency is to send two bits per symbol. PAM4 does exactly that: four amplitude levels at the same 28 GBd, across the same 14 GHz of channel. The channel does not get any worse. The signalling does.
Four levels across the same peak-to-peak swing leave three eyes, each one third the height of the single NRZ eye:
$$20\log_{10} 3 = 9.54\ \text{dB}$$
of signal-to-noise ratio given away before anything else is considered — and it is not recoverable by equalisation, because it is a property of the constellation rather than of the channel. On this channel $Q$ falls from 6.01 to 2.00, and the raw error rate rises from $10^{-9}$ to about $3\times10^{-2}$.
An error rate of one in thirty is not a link. The Reed–Solomon KP4 code, RS(544,514) over GF($2^{10}$), takes a pre-correction error rate of about $2.4\times10^{-4}$ down below $10^{-15}$, and PAM4 links are budgeted to that threshold instead of to $10^{-12}$. Even so this channel does not reach it: the required $Q$ is 3.60 and the available $Q$ is 2.00, a shortfall of 5.1 dB.
| Channel change | Loss at Nyquist | Q | Margin to the KP4 threshold |
|---|---|---|---|
| PAM4, as built | −33.3 dB | 2.00 | −5.08 dB |
| PAM4 + low-loss laminate | −28.0 dB | 2.99 | −1.60 dB |
| PAM4 + low-loss + 13″ backplane | −25.1 dB | 3.97 | +0.84 dB |
| PAM4 + all of the above, plus a receive FFE | −25.1 dB | 6.30 | +4.87 dB |
The lesson the industry learned between 2015 and 2020: PAM4 does not let you run twice the data over the channel you already have. It lets you run twice the data over a channel roughly 8 dB better than the one you already have, in exchange for a coding layer, a larger latency budget and a considerably more complicated receiver.
Nothing on slide 06 can be set from a datasheet. Manufacturing spread, temperature, the particular board and the particular connector all move the channel, so every tap in a modern link adapts. The mechanism is the least-mean-squares algorithm of the digital-filter companion deck, usually in its sign-sign form: take the sign of the error between the sampled level and the decided level, take the sign of the data in the relevant tap position, multiply, accumulate. No multiplier, no knowledge of the channel, and it converges to the same minimum of the same quadratic bowl the Wiener–Hopf equations describe.
The receiver has the error signal in hand — the difference between what it sampled and what it decided — so CTLE setting, FFE taps and DFE taps can all track the channel while data flows.
The transmitter has no idea what the receiver saw. So the standards define a link training phase: before data flows, the receiver requests increment, decrement or hold on each transmit tap over a control channel, and the transmitter reports what it did. A few milliseconds, re-run on every link-up.
The clock and data recovery loop chooses the sampling phase, and the equaliser adapts to whatever phase it is given — while the phase detector sees a waveform shaped by whatever the equaliser is currently doing. The two loops can and do fight, and a link that trains to a stable but poor setting (the wrong lock point, an eye centred half a unit interval from the best one) is a common and deeply unpleasant failure mode. Bandwidth separation between the loops is the usual defence.
The 224 Gb/s per lane generation keeps PAM4 and doubles the symbol rate again, to around 112 GBd, putting Nyquist at 56 GHz. Copper at that frequency is unforgiving, and the practical consequence is that the reach of an electrical channel keeps shrinking: a backplane of the kind modelled here is already out of the question, and even the few inches from a switch die to a front-panel cage have become a design problem rather than a piece of routing.
MLSE — instead of equalising then slicing, search for the most likely transmitted sequence given the known channel response. Worth a few dB over a DFE, at considerable digital power.
Concatenated and soft-decision schemes replacing the hard-decision Reed–Solomon codes that have served since 2017 — more coding gain, at the cost of latency and power.
Co-packaged optics move the electrical-to-optical conversion into the switch package; linear-drive pluggables remove the retimer so the host SerDes drives the optical engine directly. Both change the problem from equalising twenty inches to equalising two.
For all of that, the structure of the problem has not changed since 10GBASE-KR. There is a channel you cannot alter, a pulse response that describes it completely, a set of filters whose taps are the solution of a quadratic minimisation, and a budget that decides whether the result works. Everything in this deck transfers; only the numbers move.
| Quantity | This channel | What it tells you |
|---|---|---|
| Insertion loss at Nyquist | −33.3 dB | The headline difficulty. Beyond about −30 dB, NRZ needs every block on slide 06 |
| Cursor, unequalised | 0.199 | Fraction of the transmitted amplitude arriving on time |
| Worst-case eye, unequalised | −0.87 | Negative means some data patterns invert the decision |
| Noise enhancement of a 9-tap Rx FFE | 0.75 dB | The price of each linear tap — compare a DFE tap, which is free |
| Cursor at the slicer | 12.9 mV | Out of 800 mV launched |
| Total noise | 2.15 mV rms | Dominated by crosstalk, not by residual ISI |
| Q and margin | 6.01, −1.36 dB | Short of $10^{-12}$; slide 10 prices what recovers it |
| PAM4 penalty | 9.54 dB | Structural, not recoverable by equalisation |
1. Get the S-parameters and check them: passive, causal, reciprocal, correctly de-embedded, mixed-mode port map the right way round. 2. Look for structural damage — a via stub notch near Nyquist, a connector resonance, an unterminated branch — because no equaliser fixes a notch. 3. Compute the pulse response and read the cursor and the first few pre- and post-cursors; that tells you which blocks you need before you design any of them. 4. Build the budget and find out what actually dominates. Only then choose the equalisation — and re-check the budget afterwards, because the equaliser you added changed the noise as well as the signal.
Deck 03 of the Matrix Methods in Engineering series. Companions: Network Parameters (S, Z, Y) and Digital Filter Design.
Single-page HTML · KaTeX-rendered maths · no build step. Source on GitHub · long-form PDF