What You're Building
Evaluation frameworks don't exist in the abstract. They're purpose-built for specific models, tasks, and organizational contexts. In this capstone, you'll choose a real AI use case and design a complete evaluation system: from pre-deployment benchmarks through continuous production monitoring, covering automated metrics, human review protocols, red-teaming plans, and drift detection.
The output is a documented evaluation plan you can actually use, adapt, or share with a team. It should be specific enough that someone who hasn't read this course could implement it.
Choose Your Scenario
Select one of the following scenarios, or bring your own real use case. The frameworks you'll build are structurally similar across domains, but the specific metrics, thresholds, and risks differ substantially.
The Four-Phase Framework
Your evaluation framework should address all four phases. Work through each phase in order, since later phases depend on decisions made in earlier ones.
List the 4-6 quality dimensions that matter most for your chosen use case. For each dimension, define what "good" means in plain language, and specify a minimum acceptable threshold that, if breached, would block deployment.
Select the benchmark datasets and automated metrics you'll use before deployment. Specify any domain-specific test sets you'd need to create. Define your human evaluation protocol: rubric, reviewer criteria, sample size, and inter-annotator agreement target.
Identify the top 5 adversarial scenarios specific to your use case. For each: describe the attack type, the potential harm if successful, and the specific test prompts or strategies you'd use. Define what a "pass" looks like for each scenario.
Specify your production logging strategy, automated scoring pipeline, human review sampling rate, metric aggregation cadence, and alert thresholds. Define what events trigger each alert level, and who owns the response.
Rate how thoroughly your framework addresses each evaluation dimension. Use this as a self-check before finalizing.
Evaluation Plan Rubric
Use this rubric to self-assess your framework before finalizing it. A strong framework scores 70 or above. Anything below 50 has critical gaps that should be addressed before using the framework for real deployment decisions.
Common Framework Gaps to Avoid
- Generic metrics without thresholds: "we'll track accuracy" is not a plan. Specify the metric, the measurement method, and the minimum threshold.
- Red-teaming as a checklist: five attack types with no pass/fail criteria isn't a test plan. Define what constitutes a failure for each scenario.
- Monitoring without ownership: a dashboard that no one is responsible for watching doesn't create safety. Every alert level needs a named owner and a response SLA.
- Evaluation that ends at launch: the most common gap. A framework that only covers pre-deployment gates will miss post-deployment drift and edge cases that only appear in real traffic.
Connecting to Real Research
Your framework should reflect the current state of the field. Several concepts from this course map directly to published research you can cite when presenting your framework to stakeholders.
- For holistic multi-dimensional evaluation: HELM (Liang et al., arXiv:2211.09110) provides a template for covering accuracy, calibration, robustness, and efficiency together.
- For adversarial testing methodology: Perez et al. (arXiv:2202.03286) on prompt injection and the NIST AI Risk Management Framework provide structured approaches to systematic red-teaming.
- For domain-specific high-stakes evaluation: the MEDFIT-LLM framework (Rao, Jaggi, Naidu, IEEE RMKMATE 2025, DOI:10.1109/RMKMATE64574.2025.11042816) demonstrates how evaluation must integrate domain expert review in medical AI contexts.
- For inference cost considerations in evaluation: FrugalGPT (Chen, Zaharia, Zou, arXiv:2310.11409) shows how evaluation frameworks can account for cost-quality tradeoffs when selecting models for deployment.