AI Safety and Alignment
Course 08 · Capstone Project

Design an AI Safety Plan

Apply everything from the course to write a safety plan for a real AI use case: alignment requirements, governance tier, technical controls, and incident response.

60 min · Synthesizes all 6 modules

What You Will Build

A safety plan is a written document that a team can actually use before, during, and after deploying an AI system. It answers four questions: What can go wrong? What tier of regulatory obligation applies? What technical controls will be in place? What happens when a failure occurs? A good safety plan is specific, not generic. It names the failure modes for this use case, not AI in general.

In this capstone you will choose a scenario, work through each planning phase with the tools below, and produce a document you could take into a real deployment review. You will also see how your scenario scores across the six safety dimensions covered in this course.

Before you start Complete all six modules before attempting the capstone. The radar chart interactive scores your scenario across dimensions from each module. Missing module knowledge means missing safety dimensions in your plan.

Step 1: Choose Your Scenario

Select the AI use case your safety plan will address. Click a card to select it.

Healthcare
Clinical Decision Support
LLM summarizes records and suggests differential diagnoses for physicians.
Finance
Credit Risk Scoring
ML model ranks loan applicants and returns an approval recommendation.
HR
Resume Screening
AI filters applications and ranks candidates for human reviewer shortlisting.
Customer Support
Public-Facing Chatbot
LLM support agent answers questions about a software product.
Legal
Contract Intelligence
AI extracts key clauses and flags non-standard terms for legal review.
Education
Automated Essay Feedback
AI scores student essays and gives written feedback on a K-12 platform.

Step 2: Plan the Safety Layers

Phase 1 · Alignment
Define the intended goal and proxy metric

Write down what the system is actually optimizing for (the proxy metric) and the true goal it is meant to serve. Identify at least two ways the proxy metric could be gamed or could diverge from the true goal. This forces you to state your Goodhart's Law failure mode before it happens.

Deliverable: Intended goal, proxy metric, two divergence scenarios
Phase 2 · Governance
Classify the EU AI Act tier

Determine which EU AI Act risk tier applies to your scenario and list the compliance requirements that follow. If High Risk, name the specific Annex III category. If your organization is not in the EU, note the equivalent obligation under the jurisdiction that applies.

Deliverable: Risk tier, compliance requirements, responsible team
Phase 3 · Technical Controls
Specify the defense-in-depth layers

List at least four controls from this course that you will implement: which input validation checks, what system prompt hardening approach, what output filtering rules, whether human review is required at any stage. Be specific: "output filter for diagnostic claim types" rather than "content filter."

Deliverable: Four or more named technical controls
Phase 4 · Monitoring
Define what you log and what triggers an alert

List the metrics you will monitor in production: latency, filter trigger rate, refusal rate, human escalation rate. Define the threshold that triggers a review. Specify who receives the alert and within what timeframe.

Deliverable: Metric list, alert thresholds, notification chain
Phase 5 · Red-Teaming
Write three specific attack scenarios

For your chosen use case, write three adversarial inputs a malicious user or data source might attempt. For each, state what harm it could cause and which control layer should catch it. If no control catches it, that is a gap to fix before launch.

Deliverable: Three attack scenarios with control mapping
Phase 6 · Incident Response
Write the one-page incident response plan

Specify: severity levels (critical vs. minor), who is notified at each level, how the feature is disabled if needed, what a post-incident review looks like, and what evidence is needed before re-enabling the feature.

Deliverable: Severity matrix, notification chain, re-enable criteria
Safety Radar: Score Your Scenario Interactive

Rate your safety plan on each dimension from 1 (not addressed) to 5 (fully addressed). The radar chart updates as you adjust the sliders.

1
1
1
1
1
1
Adjust the sliders above to reflect your safety plan coverage.

Grading Rubric

Safety Plan Evaluation Criteria
Alignment analysis
Intended goal and proxy metric named explicitly. At least two divergence or gaming scenarios identified with concrete examples.
20 pts
Governance classification
Correct EU AI Act tier identified with rationale. Compliance requirements listed correctly for that tier. Jurisdictional scope addressed.
15 pts
Technical controls
At least four specific, named controls. Each control tied to a failure mode it addresses. Covers input, output, and system prompt layers.
20 pts
Monitoring plan
Metrics listed with specific thresholds. Alert notification chain named. Logging scope and retention addressed.
15 pts
Red-team scenarios
Three adversarial scenarios with realistic inputs. Each mapped to a harm and to the control layer that should catch it. Gaps acknowledged.
15 pts
Incident response
Severity levels defined. Notification chain complete. Disable mechanism identified. Re-enable criteria specified with verification step.
15 pts
Total 100 pts

Your Certificate

Click any field on the certificate below to personalise it, then print or save as PDF.

Certificate of Completion
Your Name
Click to edit
has successfully completed
AI Safety and Alignment
What is AI Safety Alignment Problem RLHF and CAI Jailbreaks Governance Practical Safety
Arjun Jaggi
AJ University · arjunjaggi.com
July 2026
Date of completion
Click any field on the certificate to edit
← Module 6: Practical Safety Course Home →

References

  1. EU AI Act : Regulation (EU) 2024/1689. eur-lex.europa.eu
  2. NIST : AI Risk Management Framework 1.0, 2023. doi.org/10.6028/NIST.AI.100-1
  3. Rao, Jaggi, Naidu : MEDFIT-LLM, IEEE RMKMATE 2025. DOI:10.1109/RMKMATE64574.2025.11042816
  4. Amodei et al. : Concrete Problems in AI Safety, arXiv:1606.06565, 2016. arxiv.org/abs/1606.06565
  5. Perez and Ribeiro : Ignore Previous Prompt, arXiv:2202.03286, 2022. arxiv.org/abs/2202.03286