Scaling and Evaluation

Why Size Alone Is Not Enough

A common assumption about AI is that bigger always means better: more parameters, more power. The reality is more nuanced. A large model trained on poor data will underperform a smaller model trained on good data. A model with many parameters but trained for too few steps has not had enough exposure to learn well. A model trained on too much data relative to its size may squeeze out every bit of signal the architecture can absorb, but no more.

Getting the balance right between model size, data volume, and training compute is one of the most important decisions in building a capable AI system. And since training a large model can take weeks and cost millions of dollars, you do not want to discover you got the balance wrong after the fact.

This is where scaling laws come in.

Scaling Laws: Predicting Capability Before You Train

In 2020, researchers published a study titled "Scaling Laws for Neural Language Models" (Kaplan et al., arXiv:2001.08361) showing something remarkable: the performance of a language model can be predicted with high accuracy before training begins, using a simple mathematical relationship between three quantities: the number of model parameters, the number of training tokens, and the total compute used.

The key finding was that loss, the measure of how wrong the model's predictions are, decreases smoothly and predictably as any of these three quantities increases. The relationship follows what mathematicians call a power law: double the compute and the loss drops by a predictable amount; double it again and it drops by roughly the same fraction again. This held across many orders of magnitude, from tiny models to the largest systems tested.

What made this practically useful is that you can run small, cheap experiments, measure the loss curve, and extrapolate to predict what a much larger model would achieve before spending the compute to build it. The recipe scales.

The Three Ingredients

The 2020 scaling laws paper established that three quantities jointly determine how capable a model will be:

• Parameters (N): the number of learned weights in the model. More parameters means more capacity to store patterns.
• Training tokens (D): how much text the model sees during training. More data means more examples to learn from.
• Compute (C): the total computational work done during training, measured in floating-point operations. Roughly, compute equals parameters multiplied by tokens.

The 2020 paper suggested that, given a fixed compute budget, you should prioritise making the model larger, even if that means training it on less data. But this turned out to be incomplete advice.

In 2022, a follow-up study known as the Chinchilla paper (Hoffmann et al., arXiv:2203.05843) reran these experiments more carefully and reached a different conclusion: for a fixed compute budget, the optimal strategy is to scale model size and data volume equally. Prior models had been significantly undertrained relative to their size — they had many parameters but had not seen enough tokens to fill that capacity.

The practical takeaway: a smaller model trained on more data can outperform a larger model trained on less, if the larger model was undertrained. Size and data are both essential, and they need to grow together.

What Does It Mean to Evaluate a Model?

Once you have trained a model, you need to know how good it is. This sounds simple. It is not.

The loss metric from training tells you how well the model predicts the next token on average across its training distribution. But what users and organisations actually care about is whether the model is useful: can it answer questions correctly, generate safe outputs, follow instructions, reason through problems, write working code?

These are different questions. A model can have excellent loss and still be useless in practice, because loss measures average prediction accuracy across a huge and varied dataset, while usefulness is specific to a task and a user population. Evaluation is the process of measuring what actually matters for the purpose you have in mind.

Benchmarks: What They Are and What They Measure

The standard way to compare AI models is through benchmarks: standardised test sets where each example has a known correct answer. A model's benchmark score is the fraction of questions it answers correctly. High score means capable model, right?

Not necessarily. Benchmarks are useful for comparing models on specific, well-defined tasks. But they come with important limitations that every practitioner should understand.

Three Ways Benchmarks Can Mislead

Benchmark saturation. When a benchmark is released, frontier models typically score far below perfect. Over time, as models improve, scores creep upward. Eventually the best models all cluster near the ceiling, and the benchmark can no longer distinguish between them. At that point it stops being informative — a model that scores 95% looks the same as one that scores 97%, even though they may differ substantially in real-world capability. HellaSwag, shown above, is a well-known example.

Data contamination. If a model's training data includes examples from a benchmark's test set, the model has seen the answers during training. It is not reasoning on the benchmark — it is recalling. This is the evaluation equivalent of the test set leakage problem from Module 3. It inflates benchmark scores without reflecting genuine capability, and it is difficult to detect because training datasets are enormous and not always fully disclosed.

Benchmark overfitting. When benchmark scores are published and used to rank models, there is pressure to optimise specifically for those benchmarks during fine-tuning. A model might be tuned to produce the right answer format on a specific test, without becoming more capable in the underlying skill the test was designed to measure. The score goes up; the real-world performance may not.

What Good Evaluation Looks Like

Researchers and practitioners increasingly combine several approaches to get a more complete picture:

Held-out task-specific sets. A custom evaluation set drawn from the actual domain the model will operate in. If you are building a model for contract review, your evaluation set should contain real contracts with known correct answers, not law-school exam questions.

Human preference ratings. For open-ended tasks where there is no single correct answer, trained raters compare model outputs side by side and indicate which they prefer. This captures qualities like helpfulness, clarity, and tone that multiple-choice benchmarks cannot.

Adversarial and edge-case testing. Deliberately constructing inputs designed to expose failure modes: ambiguous instructions, rare vocabulary, conflicting context, sensitive topics, or the kinds of prompts that real users will eventually discover and share. If you only test on easy, representative inputs, you will not discover failures until they happen in the field.

Longitudinal monitoring. Measuring model performance over time on real production inputs, not just at launch. Models can degrade as the world changes and their training data becomes stale, or as users find new patterns that expose weaknesses.