Evaluating a Fine-Tuned Model

A logistics company fine-tuned a model for order tracking queries. The training loss dropped to near zero. They deployed it, and it failed on 30% of real user questions. Training loss doesn't measure what matters. This module teaches you what does.

By the end of this module, you will be able to design an evaluation suite for a fine-tuned model, explain the difference between automatic and human evaluation, and detect catastrophic forgetting before deploying.

The Evaluation Stack

Good evaluation uses multiple complementary methods. No single metric tells the whole story.

Layer 1: Perplexity

Perplexity measures how surprised the model is by text in your held-out validation set. Lower perplexity means the model assigns higher probability to the correct tokens — it "expected" what actually appeared. Perplexity is fast to compute and tracks training progress well.

The limit: perplexity can improve even when model outputs get worse for users. A model might get better at predicting tokens in your training distribution while still failing on the edge cases that matter most in practice.

Layer 2: Task-Specific Metrics

Match the metric to the task:

• Classification: Precision, recall, F1. Don't use accuracy alone if classes are imbalanced.
• Extraction: Exact match (EM) and F1 over token overlap. Used in QA benchmarks like SQuAD.
• Generation: ROUGE (recall-oriented overlap) for summarization; BLEU for translation. Both measure n-gram overlap between generated and reference text.
• Open-ended generation: These automatic metrics often fail. Use human evaluation or LLM-as-judge instead.

Layer 3: LLM-as-Judge

Use a capable LLM (e.g., GPT-4 class) to score your fine-tuned model's outputs on a rubric: helpfulness, accuracy, tone, and format. This scales better than human evaluation and correlates well with human preference when the judge model is calibrated carefully (Zheng et al., arXiv:2306.05685).

Layer 4: Human Evaluation

For high-stakes applications, there is no substitute. Have real users (or domain experts) rate outputs on a 1–5 scale. Sample 100–200 representative inputs. This is expensive but provides ground truth that automatic metrics can't give you.

Catastrophic Forgetting: The Hidden Trap

Fine-tuning on a narrow dataset can cause the model to forget what it knew from pre-training. This is called catastrophic forgetting. The model gets better at your task but worse at everything else. In a customer support model, this might mean the model starts refusing to answer general questions it handled confidently before fine-tuning.

How to detect it: run a general-purpose benchmark (MMLU, HellaSwag, or a set of diverse test questions) on both the base model and your fine-tuned model. If scores drop significantly (more than 5 percentage points), your fine-tuning is too aggressive. Reduce epochs, lower learning rate, or use LoRA, which is less likely to cause forgetting than full fine-tuning.

from datasets import load_dataset
from evaluate import load
import torch

def evaluate_model(model, tokenizer, test_data, general_benchmark):
    """
    Run two-track evaluation: task-specific + forgetting check.
    """
    results = {}

    # Track 1: Task-specific F1 on your domain test set
    rouge = load("rouge")
    task_preds = []
    task_refs = []

    model.eval()
    with torch.no_grad():
        for ex in test_data:
            inputs = tokenizer(ex["instruction"], return_tensors="pt").to(model.device)
            outputs = model.generate(**inputs, max_new_tokens=128)
            pred = tokenizer.decode(outputs[0], skip_special_tokens=True)
            task_preds.append(pred)
            task_refs.append(ex["output"])

    rouge_scores = rouge.compute(predictions=task_preds, references=task_refs)
    results["task_rouge_l"] = rouge_scores["rougeL"]

    # Track 2: General capability check (catastrophic forgetting detection)
    # Run a subset of MMLU or your own general questions
    general_correct = 0
    for q in general_benchmark[:100]:
        inputs = tokenizer(q["question"], return_tensors="pt").to(model.device)
        with torch.no_grad():
            outputs = model.generate(**inputs, max_new_tokens=50)
        pred = tokenizer.decode(outputs[0], skip_special_tokens=True)
        if q["answer"].lower() in pred.lower():
            general_correct += 1

    results["general_accuracy"] = general_correct / 100
    results["forgetting_risk"] = "HIGH" if results["general_accuracy"] < 0.6 else "LOW"

    return results

A Minimal Evaluation Suite You Can Build Today

You don't need a research team or infrastructure to run useful evaluations. Here's a practical checklist:

1. Hold out 15% of your data before fine-tuning. Never train on it. Use it to measure task performance.
2. Compile 30 general questions covering topics the base model handles well (geography, basic math, factual questions). Run these on both models and compare.
3. Run 20 adversarial inputs: edge cases, unusual phrasings, inputs just outside your training distribution. These reveal gaps that average metrics hide.
4. Score 20 outputs with LLM-as-judge on: accuracy (1–5), format correctness (1–5), and appropriateness for your use case (1–5).
5. Compare distributions rather than just averages. A model with average F1 of 0.82 might still fail badly on a specific subcategory. Always segment results.