Every AI vendor cites benchmark numbers. Almost none of those benchmarks measure what you actually need. This module maps the major benchmarks: what each one tests, what it misses, and how to tell whether a benchmark score is meaningful for your use case or is just a marketing number dressed in research clothing.
The AI research community has developed hundreds of evaluation benchmarks, but five have become the standard references cited in model technical reports and vendor comparisons. Understanding what each one was designed to measure, and what it systematically cannot measure, is the foundation of intelligent benchmark reading.
| Benchmark | What it measures | Format | What it misses |
|---|---|---|---|
| MMLU | Factual knowledge across 57 academic subjects including law, medicine, mathematics, and history | 4-choice multiple choice, 14,000+ questions | Reasoning quality, instruction following, helpfulness in conversation, safety |
| BIG-bench | Novel reasoning tasks designed to be harder than what models see in training: logical inference, analogical reasoning, arithmetic | 204 tasks, mixed formats | Real-world task performance, domain expertise, long-form generation quality |
| TruthfulQA | Whether models repeat common human misconceptions and false beliefs when answering questions | 817 questions across 38 categories | Knowledge breadth, reasoning ability, instruction following; only tests a narrow honesty dimension |
| MT-Bench | Multi-turn instruction following ability: writing, reasoning, math, coding, extraction across two-turn conversations | 80 questions, GPT-4 rated 1-10 | Factual accuracy, safety, domain-specific tasks; rating quality depends on judge model |
| HELM | Holistic view across multiple dimensions: accuracy, calibration, robustness, fairness, efficiency, and more simultaneously | 42 scenarios across multiple task types | Custom domain performance, real deployment conditions, latest task types |
MMLU (Hendrycks et al., arXiv:2009.03300) tests whether a model knows facts across a wide range of academic subjects. It is useful for understanding whether a model has absorbed broad world knowledge. It tells you almost nothing about whether the model can follow complex instructions, generate useful long-form content, or behave safely under pressure. A model that scores 88% on MMLU might still write confusing instructions, give unhelpful answers to ambiguous questions, and fail on tasks that require multi-step planning.
BIG-bench (Srivastava et al., arXiv:2206.04615) was designed to be hard enough that contemporary models would fail at it, creating a useful signal about reasoning ability beyond pattern matching. Some of its tasks have already been largely solved by recent models. The "hard" subset, BIG-bench Hard, focuses on tasks where chain-of-thought reasoning provides the most advantage, making it a useful probe of whether a model can think step by step rather than recall.
TruthfulQA (Lin et al., arXiv:2109.07958) tests a very specific and important property: does the model resist the temptation to repeat false beliefs that many humans hold? The questions are designed so that a model that confidently mimics human answers would score poorly. A model that scores well on TruthfulQA is less likely to confidently state things that are wrong, but this benchmark does not tell you whether the model hallucinates facts it does not know, which is a different failure mode entirely.
MT-Bench and Chatbot Arena (Zheng et al., arXiv:2306.05685) brought human preference signals into evaluation. MT-Bench uses an LLM judge to rate responses on a fixed set of multi-turn conversations. Chatbot Arena uses crowdsourced human ratings from real conversations. These approaches capture dimensions like helpfulness, clarity, and appropriate response length that automated metrics miss. The limitation is that human preference is not the same as correctness: a model can be preferred by users while still being factually inaccurate on technical questions.
HELM (Liang et al., arXiv:2211.09110) is the most ambitious attempt to evaluate models holistically rather than on a single axis. It tests 30 models across 42 scenarios and evaluates on seven metrics simultaneously: accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency. The practical limitation of HELM is that its multi-dimensional framework makes it hard to summarize in a single number, which means it rarely shows up in vendor marketing.
Benchmark contamination occurs when a model's training data includes text from the benchmark itself: the questions, the answer options, the correct answers, and sometimes even analyses of the benchmark that reveal which answers are correct.
This is not intentional cheating. It happens because the internet contains enormous amounts of text about these benchmarks, including published papers, blog posts, and educational materials that discuss the benchmark questions and their correct answers. A model trained on a large web crawl will likely have seen some fraction of this material.
The practical consequence is that benchmark scores for models trained on internet data include a contamination component that is difficult to separate from true capability. Researchers have developed techniques to detect contamination, including checking whether models perform unusually well on benchmark questions compared to paraphrased versions of the same questions, or whether they perform better on well-known benchmarks than on equivalent but less-published benchmarks covering the same material.
BLEU (Papineni et al., ACL 2002) and ROUGE are automated metrics for evaluating text generation tasks like translation, summarization, and paraphrase. They measure overlap between the model's output and one or more reference texts.
BLEU computes the fraction of n-grams in the model's output that also appear in the reference text, with a penalty for outputs that are too short. A BLEU score of 1.0 means the output exactly matches the reference. Real translation systems score meaningfully below 1.0 on standard benchmarks, with higher scores indicating better translation quality within a specific language pair and domain.
ROUGE works in the opposite direction: it measures the fraction of n-grams in the reference that appear in the model's output, capturing recall rather than precision. ROUGE-L measures the longest common subsequence, which is more tolerant of word reordering than exact n-gram overlap.
Both metrics have well-documented limitations. They reward lexical similarity but cannot evaluate semantic accuracy, fluency, or appropriateness. A translation that uses different but equivalent vocabulary will score lower than a word-for-word translation that happens to match the reference phrasing even if the different vocabulary is clearer to native speakers. For this reason, BLEU and ROUGE are used as fast proxies during development, not as final quality judges. Module 3 covers human evaluation as the complementary method that captures what BLEU and ROUGE miss.
Select the dimensions that matter for your use case. The canvas shows which benchmarks cover your requirements and where the gaps are.
# Evaluate a model on a subset of MMLU categories
# Dataset available at: huggingface.co/datasets/cais/mmlu
from datasets import load_dataset
from typing import Callable
def evaluate_mmlu_subset(
model_fn: Callable[[str], str],
categories: list = ["anatomy", "clinical_knowledge", "medical_genetics"],
num_examples: int = 50
) -> dict:
"""
Run a model against MMLU multiple-choice questions.
model_fn: takes a prompt string, returns the model's answer string
"""
results = {}
for cat in categories:
dataset = load_dataset("cais/mmlu", cat, split="test")
correct = 0
total = min(num_examples, len(dataset))
for i in range(total):
item = dataset[i]
prompt = f"""Question: {item['question']}
A. {item['choices'][0]}
B. {item['choices'][1]}
C. {item['choices'][2]}
D. {item['choices'][3]}
Answer with A, B, C, or D only:"""
response = model_fn(prompt).strip().upper()
answer_key = "ABCD"[item['answer']]
if response and response[0] == answer_key:
correct += 1
results[cat] = {
"accuracy": correct / total,
"correct": correct,
"total": total
}
overall = sum(v["correct"] for v in results.values())
total_q = sum(v["total"] for v in results.values())
results["overall"] = {"accuracy": overall / total_q, "correct": overall, "total": total_q}
return results
# Example usage:
# scores = evaluate_mmlu_subset(my_model_fn, categories=["clinical_knowledge"])
# print(f"Clinical knowledge accuracy: {scores['clinical_knowledge']['accuracy']:.1%}")Look up the technical report or model card of the AI system your team uses most often. Find the benchmark scores listed. For each benchmark, ask two questions: (1) Is this benchmark testing a dimension that actually matters for your specific use case? (2) When was the benchmark released, and is it old enough that the model may have seen training data based on it? Write your answers in three sentences each. This exercise will immediately reveal how much of the vendor's benchmark marketing is relevant to your situation.