Evaluation & Benchmarks — How Do We Know It Works?

The Five Publications

• MMLU — Measuring Massive Multitask Language Understanding
• HumanEval — Evaluating LLMs Trained on Code (pass@k)
• BIG-bench — Beyond the Imitation Game
• HELM — Holistic Evaluation of Language Models
• LLM-as-a-Judge — MT-Bench & Chatbot Arena
• What to Take Away

MMLU — Measuring Massive Multitask Language Understanding

Hendrycks, Burns, Basart, Zou, Mazeika, Song & Steinhardt — 2020 ICLR 2021 / arXiv:2009.03300

The problem

By 2020, models were saturating narrow benchmarks (GLUE, SuperGLUE) that tested a handful of linguistic skills. There was no single test that asked: does the model actually know things across the breadth of human knowledge a well-read person is expected to have? Existing evals could not distinguish a model with shallow pattern-matching from one with genuine multidisciplinary understanding.

The contribution

MMLU is a four-way multiple-choice exam spanning 57 subjects — from elementary mathematics and US history to college-level law, professional medicine, abstract algebra, philosophy and machine learning. Questions are drawn from real exams and practice tests. Crucially, models are evaluated zero-shot and few-shot (typically 5-shot), never fine-tuned on the task, so the score reflects knowledge acquired during pre-training. Random chance is 25%; the authors reported GPT-3 at ~43.9%, against ~34.5% for unspecialised (Mechanical Turk) humans and an estimated ~89.8% for domain experts, exposing a large gap to expert level.

Why it matters to a practising engineer

MMLU became the headline number quoted on every model card — if you compare base models, you are almost certainly comparing MMLU. But it is now partly saturated: frontier models score in the high 80s, compressing the signal and making small differences noisy. It is also sensitive to prompt format and answer-parsing, and has known label errors. MMLU-Pro (2024) responded with ten options, harder reasoning-heavy questions and reduced answer leakage. Treat MMLU as a coarse pre-training sanity check, not a fine-grained ranking tool.

HumanEval — Evaluating Large Language Models Trained on Code

Chen et al. (OpenAI) — 2021 arXiv:2107.03374   Codex

The problem

Code generation cannot be judged by surface text similarity to a reference solution — two correct programs can look nothing alike, and a near-identical program can be subtly broken. BLEU-style match metrics are actively misleading for code. The field needed a metric grounded in functional correctness: does the generated program actually run and produce the right answers?

The contribution

The paper introduced HumanEval: 164 hand-written Python problems, each a function signature plus docstring, accompanied by an average of ~7.7 hidden unit tests. A sample is correct only if it passes every test. To score stochastic sampling fairly, the paper popularised the pass@k metric: the probability that at least one of k sampled completions is correct. Naively estimating this by drawing exactly k samples is high-variance, so they generate n > k samples, count the c that pass, and use an unbiased estimator.

import numpy as np

def pass_at_k(n: int, c: int, k: int) -> float:
    # n = total samples generated, c = number correct, k = the k in pass@k
    # 1 - C(n-c, k) / C(n, k)  =  P(at least one of k drawn samples passes)
    if n - c < k:
        return 1.0
    return 1.0 - np.prod(1.0 - k / np.arange(n - c + 1, n + 1))

The product form avoids overflow from large binomial coefficients and is numerically stable. Codex (a GPT model fine-tuned on public code) reached 28.8% pass@1 and ~70% pass@100 on HumanEval — the gap between the two showing that the model often can solve a problem if allowed many attempts, which motivated later work on sampling and verification.

BIG-bench — Beyond the Imitation Game

Srivastava et al. (450+ authors, 130+ institutions) — 2022 arXiv:2206.04615

The problem

Benchmarks were being saturated faster than they could be built, and each tested a narrow slice of capability. Worse, hand-curated tasks reflected the blind spots of a small group of researchers. The community needed a benchmark that was broad, diverse, and deliberately hard — designed to probe what current models cannot yet do, and to keep being informative as models scale.

The contribution

BIG-bench (Beyond the Imitation Game Benchmark) is a collaborative suite of 200+ tasks contributed by the community, spanning logic, mathematics, common-sense reasoning, linguistics, social bias, code, and many tasks invented specifically to be difficult for language models. It studies how performance scales with model size, and identifies emergent tasks where capability appears suddenly past a scale threshold and flat tasks where scale barely helps.

Its most enduring legacy is the BIG-Bench Hard (BBH) subset: 23 tasks on which models at the time of release scored below the average human rater. BBH became a standard reasoning benchmark precisely because it is where chain-of-thought prompting helps most — CoT flips several BBH tasks from below-random to strongly above-human, a result that helped cement step-by-step prompting as a default technique.

HELM — Holistic Evaluation of Language Models

Liang et al. (Stanford CRFM) — 2022 arXiv:2211.09110

The problem

Most evaluation reported a single accuracy number on a single task for a single model. That hides everything a deployer actually cares about: is the model calibrated? Robust to typos and paraphrase? Fair across demographics? Does it emit toxic or biased text? How efficient is it? And no two papers evaluated the same models on the same conditions, so cross-model comparison was apples-to-oranges.

The contribution

HELM reframes evaluation as a scenarios × metrics matrix, evaluated densely and transparently. It defines a taxonomy of scenarios (task + domain + who-it-is-for) and measures seven metric categories on every scenario simultaneously, rather than reporting accuracy alone. It then ran the same standardised conditions across dozens of models, publishing every raw prediction for reproducibility — making it the first large-scale, like-for-like, multi-metric audit of the model landscape.

Why it matters to a practising engineer

HELM is the template for a real model selection process. A model that wins on accuracy may lose badly on calibration or cost — both decisive in production. The living leaderboard (and its later spin-offs HELM Lite, HEIM for images, and HELM Instruct) gives a reproducible, multi-axis comparison you can cite. The deeper lesson: pick the metrics that map to your deployment risks and measure all of them at once, not just the headline.

LLM-as-a-Judge — MT-Bench & Chatbot Arena

Zheng et al. (LMSYS) — 2023 NeurIPS 2023 / arXiv:2306.05685

The problem

For open-ended chat, multiple-choice accuracy is meaningless — quality is about helpfulness, style, instruction-following and coherence over multiple turns. Human preference labelling is the gold standard but is slow and expensive, and it does not scale to the rapid cadence of model releases. Can a strong LLM stand in as an automatic judge, and if so, how trustworthy is it?

The contribution

The paper makes the case for LLM-as-a-judge — using a strong model (GPT-4) to grade answers — and shows it reaches ~80%+ agreement with human preferences, comparable to human–human agreement. Critically, it does not just advocate the method, it documents the judge's systematic biases and proposes mitigations. It introduces two complementary tools: MT-Bench, 80 curated multi-turn questions across eight categories scored by the judge, and Chatbot Arena, a crowdsourced platform where anonymous users chat with two models side-by-side, vote a winner, and the pairwise outcomes are aggregated into an Elo leaderboard.

Why it matters to a practising engineer

LLM-as-a-judge is now the workhorse of internal evals, RLHF/RLAIF reward modelling, and CI gates for prompt and model changes — it is fast, cheap and scalable. But it is only trustworthy if you actively neutralise its biases: randomise position, control length, avoid same-family judging, and validate against a small human-labelled set. Chatbot Arena, meanwhile, remains the most-watched real-world preference signal — though its scores are confounded by traffic mix, style preferences and prompt distribution.

What to Take Away

• No single number is enough. The five papers form a ladder — knowledge (MMLU), functional correctness (HumanEval), hard capability (BIG-bench/BBH), holistic multi-metric audit (HELM), and open-ended human preference (Arena/judge). A serious evaluation samples across the whole ladder.
• Match the metric to the modality. Multiple-choice for knowledge, unit tests for code, pairwise preference for chat. Using BLEU on code or accuracy on open-ended chat is a category error.
• pass@k separates capability from selection. A large pass@1-to-pass@100 gap means correct answers already exist in the distribution — invest in reranking, search and self-repair, not just a bigger model.
• Saturation and contamination erode benchmarks. MMLU is partly saturated (hence MMLU-Pro); BIG-bench has leaked into training data. Always note training cutoff, shot count and harness when quoting a score.
• Measure what you will ship on. HELM's lesson: evaluate accuracy, calibration, robustness, fairness, toxicity and cost together, and store raw outputs so the eval is reproducible.
• LLM-as-judge is powerful but biased. Neutralise position and verbosity bias, avoid self-enhancement, and anchor to a small human-labelled set before you trust it as a CI gate.