One loop, repeated: a batch of text, a forward pass, a loss, a backward pass, an update. What next-token cross-entropy measures, why its gradient is p − y, and what a real loss curve looks like, measured on a 5,376-parameter decoder that trains in a browser.
This deck follows chapter 1, the training loop and chapter 3, the objective of the Training site, where the Decoder site’s model trains in your browser. Every measured number here comes from the site’s results.md: seeded runs of a NumPy reference trainer whose backward pass is checked against PyTorch.
Training is one loop, repeated: take a batch of text, run the model forward, score its guesses with a loss, send the gradient of that loss backward through every weight, and nudge each weight downhill (update). One turn is one training step.
export function trainStep(tr: Trainer, batch?: Batch): StepLog {
const { cfg } = tr;
const b =
batch ?? sampleBatch(tr.train, cfg.batch, cfg.model.seq_len, tr.rng);
const { loss, grads } = lossAndGrads(
tr.params,
cfg.model,
b.inputs,
b.targets,
);
const { norm, scale } =
cfg.clip === null
? { norm: NaN, scale: 1 }
: clipByGlobalNorm(grads, cfg.clip);
const lr = lrAt(tr.t, { ...cfg.schedule, total: cfg.steps });
optimizerStep(tr.opt, tr.params, grads, lr);
const log = { step: tr.t, lr, loss, gradNorm: norm, clipScale: scale };
tr.t += 1;
return log;
}In PyTorch the same four steps are loss = model(x, y), loss.backward(), clip_grad_norm_(…) and opt.step(). They are the heart of every training loop, from this toy to a model trained on thousands of accelerators; decks 03 and 04 open up the backward pass and the optimiser.
The model is the Decoder site’s, with the same random initial weights: the one the Decoder site runs one forward pass at a time. The Training site trains it.
The default run: AdamW (β1 0.9, β2 0.99, weight decay 0.01), learning rate 0.01 with 50 warmup steps then a cosine to a tenth of it, gradient clipping at norm 1, 2,000 steps. It took 8.3 s in NumPy on a CPU. Small enough to follow every number; large enough to show the real phenomena.
A training example is a window of 9 consecutive characters. The first 8 are the input; the target at each position is the character after it. Feed “alice wa” in and the model returns 8 rows of 64 logits, one per position. Thanks to the causal mask, row \(i\) saw only characters \(1 \dots i\), so each row is a genuine prediction of character \(i+1\).
The model always predicts from the true text so far, never from its own guesses: teacher forcing. So all 8 positions are scored at once, in one forward pass. This is what makes transformers efficient to train, compared with generating, where each token must wait for the one before. The chain rule of probability makes it exact:
$$\begin{aligned}&\log p_\theta(w_2,\dots,w_{9} \mid w_1) \\ &\quad= \sum_{i=1}^{8} \log p_\theta(w_{i+1} \mid w_{1..i})\end{aligned}$$The loss is the mean, over every predicted position, of minus the log of the probability the model gave the character that actually came next: the next-token cross-entropy.
$$L(\theta) = -\frac{1}{BS}\sum_{b=1}^{B}\sum_{i=1}^{S} \log p_\theta\big(y_{b,i} \mid x_{b,1..i}\big)$$Probability 1 costs nothing; 1/2 costs ln 2 ≈ 0.69; 1/64 costs ln 64 = 4.1589, the score of a uniform guess. Being confidently wrong is expensive without limit, so the model keeps some probability on everything plausible. Averaged over the text, the loss is the number of nats needed to encode each next character, and minimising it is maximum-likelihood estimation.
With logits \(z\) and \(p = \operatorname{softmax}(z)\), the loss at one position and its gradient are
$$\begin{gathered}\ell = -\log p_y = -z_y + \log \sum_j e^{z_j} \\ \frac{\partial \ell}{\partial z_k} = p_k - \mathbf 1[k = y]\end{gathered}$$The right character’s logit is pushed up by \(1 - p_y\); every other logit is pushed down by its own probability. The pushes sum to zero. When the model is already right, the gradient vanishes; when it is confidently wrong, it is large.
\(\log\sum_j e^{z_j}\) is evaluated as \(m + \log\sum_j e^{z_j - m}\) with \(m = \max_j z_j\), so no exponential overflows. For the mean over \(N = BS\) positions each row is divided by \(N\). Everything in backpropagation (deck 03) starts from this vector.
return probs.map((p, i) =>
p.map((v, j) => (v - (j === targets[i] ? 1 : 0)) / n),
);Label smoothing replaces the one-hot target with \((1-\varepsilon)\mathbf 1_y + \varepsilon/V\) (Szegedy et al. 2015, §7); it is common in translation and image classification, while language-model pretraining usually uses the plain loss, as here.
At step 0 every character is about equally likely: validation loss 4.1774, against ln 64 = 4.1589. By step 200 it is 2.9446, close to 3.0000, the unigram entropy of the text: what a model scores if it knows only how common each letter is. Then it sits there for several hundred steps. A model this small, started this small, first learns letter frequencies through its output layer, and only later finds that the previous characters (which attention must route to the current position) tell it far more. Once it does, the loss falls again: 2.7777 at step 1,000 and 2.2993 at step 2,000.
The training loss is measured on each step’s own batch: noisy, because every batch differs. The validation loss scores a fixed set of windows from the held-out end of the book, text the model never trains on. If training loss kept falling while validation loss rose, the model would be memorising (overfitting). Here they stay together: the run sees each training character about 2.0 times, far too little for 5,376 weights to memorise 127,837 characters.
\(e^{L}\) turns a loss back into a count: a model with loss \(L\) is as unsure as a uniform choice among \(e^{L}\) tokens. The run ends at a validation perplexity of 10.0, against 20.1 for letter frequencies alone and 64 for a uniform guess. Perplexities compare only with the same tokeniser: a BPE token covers more text than a character.
| Step | Validation loss |
|---|---|
| 0 | 4.1774 |
| 100 | 2.9702 |
| 200 | 2.9446 |
| 500 | 2.9604 |
| 800 | 2.9548 |
| 900 | 2.8593 |
| 1,000 | 2.7777 |
| 1,100 | 2.6193 |
| 1,200 | 2.4653 |
| 1,500 | 2.3685 |
| 2,000 | 2.2993 |
Everything random is seeded: the weights (seed 42, the Decoder’s), the order of the batches (data seed 1) and the sampler (seed 7). The same seeds give the same run on any machine, which is why the site’s browser run lands on the recorded one.
A different data seed gives a different curve. The default settings over data seeds 1, 2 and 3 end at 2.299, 2.354 and 2.178: mean ± standard deviation 2.277 ± 0.090 (results.md §6a).
A difference smaller than the seed-to-seed spread is not a difference, and a result from a single seed is a hypothesis until it is repeated. Many of these runs are also chaotic: when the model leaves the plateau depends on the last bits of its arithmetic, so recomputing a run on another CPU moved single seeds’ final losses by up to about 0.35 (results.md, “how to read these numbers”). Every comparison in this series is three seeds, mean ± standard deviation.
Start from “alice ”, predict a distribution over the next character, draw one at temperature 0.8 (as in the Decoder’s sampling chapter), append it, repeat. Sampler seed 7, 120 characters (results.md §4):
alice ae$(814q -rk+8nyt9t$q[mjkt +f{ ?"$?}=clx2bg+y25@("wxjj{vm;wc"} =:}i8:]#<!$$ /a7v)
$ =2;- uqab@=;b?!)%s%:*?=j%7.k0o/>-:%alice ad, onle the pang hul cag soe she."
"ainnd the wos id welund wit war to srur ang poy the shecavt as twe, thok" pand the;It has learnt spaces, common short words and punctuation, but not sense: with 8 characters of context and about five thousand weights, it cannot. Three steps of the loop teach the mechanism; a couple of thousand show the language arriving.
The site’s trainer is a NumPy reference with a hand-written backward pass (deck 03), checked against PyTorch, and a TypeScript port that trains in a Web Worker and replays the reference: the first 200 steps of the default run to 10−9, the whole run to within 10−7 of the recorded weights.
| Check (results.md §3) | Measured max error | Tolerance |
|---|---|---|
| Backward = PyTorch autograd (perturbed weights) | 9.4×10−17 (relative) | 10−12 |
| Backward = central differences, 552 entries | 7.6×10−10 | 10−8 + 10−6|g| |
| SGD = torch.optim, 25 steps | 0 | 10−12 |
| Adam = torch.optim, 25 steps | 2.2×10−16 | 10−12 |
| AdamW = torch.optim, 25 steps | 1.1×10−16 | 10−12 |
| Cosine schedule = CosineAnnealingLR | 3.5×10−18 | 10−15 |
CI regenerates every fixture and result and fails on any difference beyond its stated rule, so a number on these slides is one the repository still reproduces.
Animated: chapter 1, the training loop (the model trains live) · chapter 3, the objective · next: deck 02, data and tokenisation