AI Safety and Alignment
All Levels 40 min Module 5 of 6
Module 5 of 6

AI Governance and Policy

The EU AI Act is not bureaucracy. It is a definition of what companies are legally responsible for when they deploy AI systems that affect people. Understanding it is not a compliance exercise: it is a map of which risks regulators believe matter most and who is expected to manage them. This module explains the key frameworks in plain language and shows what each requires in practice.

By the end of this module you will be able to

The EU AI Act: A Risk-Based Approach

The EU AI Act (Regulation EU 2024/1689), which entered into force in August 2024, establishes a four-tier risk classification for AI systems. The tier determines what compliance obligations apply. Systems in higher risk tiers face more stringent requirements. Systems in the highest risk tier are banned entirely.

The framework is risk-based rather than technology-based: it is not about what the system is built with, but about what it is used for and what harms it could cause. An AI system for playing chess in a game app faces minimal regulation. The same underlying technology used to make decisions about who receives social benefits falls under high-risk requirements.

Tier 1 : Unacceptable Risk
Prohibited Practices
These uses are banned outright. Examples include: subliminal manipulation that affects behavior against a person's will, social scoring by public authorities, real-time remote biometric identification in public spaces (with narrow exceptions), AI that exploits vulnerabilities of specific groups. Violations: up to 7% of global annual turnover (EU AI Act Article 5, penalties under Article 99).
Tier 2 : High Risk
Strict Compliance Requirements
Applications with significant potential harm that may be deployed with appropriate safeguards. Includes: AI in critical infrastructure, educational or vocational training assessment, employment and HR decisions, access to essential services (credit scoring, insurance), law enforcement, migration and asylum management, administration of justice. Requires: conformity assessment, risk management system, technical documentation, data governance, human oversight, accuracy and robustness testing, EU database registration. Violations of provider/deployer obligations: up to 3% of global annual turnover.
Tier 3 : Limited Risk
Transparency Obligations
AI systems that interact with humans or generate content must be transparent about their AI nature. Chatbots must disclose they are not human when a user asks. Deepfakes and synthetic media must be labeled. No conformity assessment required. The transparency obligation is the key requirement.
Tier 4 : Minimal Risk
Voluntary Codes of Conduct
The vast majority of AI applications: spam filters, AI-powered video games, recommendation systems, most productivity tools. No mandatory requirements beyond general law. Organizations encouraged to adopt voluntary codes of practice but not required to.
Fig 5 · EU AI Act Risk Pyramid
MINIMAL RISK Most AI apps : voluntary codes only LIMITED RISK Chatbots, deepfakes : transparency only HIGH RISK HR, credit, healthcare : full compliance PROHIBITED Banned 7% turnover 3% turnover EU AI Act, Regulation (EU) 2024/1689 : directional illustration

What High-Risk AI Systems Must Do

High-risk AI systems under the EU AI Act face the most specific requirements. These are summarized from the Act's technical requirements (Article 9-15):

NIST AI Risk Management Framework

The National Institute of Standards and Technology (NIST) published the AI Risk Management Framework (AI RMF 1.0) in January 2023. It is voluntary for US organizations but widely used as a practical implementation guide alongside regulatory requirements globally.

GV
Govern
Establish organizational policies, roles, and processes for AI risk management. Culture, accountability structures, and risk tolerance definitions. This function runs continuously and enables all others.
MP
Map
Identify and categorize AI risks in context. Understand the use case, stakeholders, potential impacts, and risk categories. Creates the foundation for measurement and management decisions.
ME
Measure
Analyze and assess AI risks using quantitative and qualitative methods. Evaluate trustworthiness characteristics: fairness, explainability, privacy, security, reliability. This is where evaluation practice (see our Evaluating AI course) applies directly.
MG
Manage
Prioritize and address risks based on measurement results. Implement risk responses, monitor residual risk, and maintain documentation. Includes deployment decisions, ongoing monitoring, and incident response.

ISO/IEC 42001: The AI Management System Standard

ISO/IEC 42001:2023 is the first international standard specifically for AI management systems. It follows the high-level structure common to ISO management system standards (similar in format to ISO 27001 for information security), which means organizations with existing management system certifications can integrate it into their existing compliance programs.

The standard addresses: context and scope of AI systems, leadership and organizational commitment to responsible AI, planning including risk and opportunity assessment, support including documentation and training, operational controls for AI development and deployment, performance evaluation, and improvement processes.

ISO 42001 is particularly relevant for organizations that want an auditable, third-party-certifiable framework for AI governance. It complements the NIST AI RMF (which is more prescriptive on risk categories) and provides an operational structure for demonstrating governance maturity to customers, regulators, and partners.

Interactive 5: EU AI Act Risk Tier Classifier Try it

Select an AI use case to see which EU AI Act risk tier it most likely falls into and what that means in practice.

Python · AI Risk Assessment Checklist
# A simplified EU AI Act risk tier assessment.
# Real compliance requires legal review and formal conformity assessment.

from dataclasses import dataclass, field
from typing import List

@dataclass
class AISystemProfile:
    name: str
    domain: str               # e.g. "healthcare", "hr", "finance"
    makes_consequential_decisions: bool
    affects_fundamental_rights: bool
    uses_biometrics: bool
    is_safety_critical: bool
    has_human_oversight: bool
    has_explainability: bool

def classify_eu_ai_act_tier(profile: AISystemProfile) -> dict:
    """
    Simplified EU AI Act tier classification.
    Returns tier (1-4) and key obligations.
    Does NOT constitute legal advice.
    """
    # Tier 1: Prohibited (simplified check)
    if profile.uses_biometrics and profile.domain == "law_enforcement_realtime":
        return {
            "tier": 1,
            "label": "PROHIBITED",
            "action": "Cannot be deployed under EU AI Act Article 5"
        }

    # Tier 2: High risk
    high_risk_domains = ["healthcare", "hr", "credit", "education", "justice"]
    if (profile.domain in high_risk_domains
            and profile.makes_consequential_decisions):
        obligations = [
            "Register in EU AI Act database",
            "Conformity assessment required",
            "Risk management documentation",
            "Human oversight mechanism",
            "Accuracy and bias testing",
            "Post-market monitoring plan"
        ]
        return {
            "tier": 2,
            "label": "HIGH RISK",
            "obligations": obligations,
            "penalty": "Up to 3% of global annual turnover"
        }

    # Tier 3: Limited risk (transparency)
    if profile.domain in ["chatbot", "synthetic_media"]:
        return {
            "tier": 3,
            "label": "LIMITED RISK",
            "obligations": ["Disclose AI nature to users", "Label synthetic content"],
            "penalty": "Transparency violation penalties"
        }

    # Tier 4: Minimal risk
    return {
        "tier": 4,
        "label": "MINIMAL RISK",
        "obligations": ["Voluntary code of conduct", "General applicable law"],
        "penalty": "None specific to AI Act"
    }


# Example usage
hiring_ai = AISystemProfile(
    name="Resume Screener v2",
    domain="hr",
    makes_consequential_decisions=True,
    affects_fundamental_rights=True,
    uses_biometrics=False,
    is_safety_critical=False,
    has_human_oversight=True,
    has_explainability=False
)

result = classify_eu_ai_act_tier(hiring_ai)
print(f"System: {hiring_ai.name}")
print(f"Tier: {result['label']}")
print(f"Key obligations: {result.get('obligations', 'None')}")
Try this

List three AI systems your organization uses or is evaluating. For each, classify them according to the EU AI Act tier using the criteria in this module: does it make consequential decisions about people? In what domain? Does it use biometrics? This classification takes 15 minutes and immediately clarifies which systems need compliance attention before the Act's implementation deadlines. High-risk AI system obligations began phasing in from August 2026 for new systems.

Does the EU AI Act apply to companies outside the EU?
Yes, with extraterritorial reach similar to the GDPR. The Act applies if the output of an AI system is used in the EU, regardless of where the system or its developer is located. A US company deploying an AI hiring tool used to make decisions about EU-based employees would fall under the Act's scope for that use case. Companies that provide AI systems to EU-based customers or operators also fall within scope if those systems are used to make consequential decisions about EU residents. Many organizations outside the EU that do business there are already mapping their AI systems for compliance.
Knowledge check
Under the EU AI Act, an AI system used to automatically screen job applications and rank candidates falls into which risk tier?
Correct. Employment and HR decisions are explicitly listed as high-risk AI uses under the EU AI Act (Annex III). A hiring AI must meet requirements for risk management documentation, technical documentation, human oversight, and accuracy testing, among others.
Not quite. Employment decisions are explicitly categorized as high-risk under EU AI Act Annex III. This category includes systems used for recruitment, selection, and promotion decisions. The system must meet Tier 2 compliance requirements: risk management, technical documentation, human oversight, and accuracy testing.
The NIST AI Risk Management Framework organizes AI risk management into four functions. Which one focuses on establishing organizational policies, culture, and accountability for AI risk?
Correct. The Govern function in the NIST AI RMF addresses organizational culture, accountability, roles, and policies for AI risk management. It runs continuously and enables all other functions. Map identifies risks, Measure assesses them, and Manage addresses them.
Not quite. The Govern function is the one that addresses organizational policies, culture, and accountability. Map identifies and categorizes AI risks in context. Measure analyzes and assesses those risks. Manage prioritizes and addresses risks based on measurement results.
Before you go
Reflection: Classify the most consequential AI system your organization currently uses or is considering. Which EU AI Act tier does it fall into? What obligations does that tier create?
You might also like
Was this module helpful?

Work with Arjun on EU AI Act compliance strategy and AI governance frameworks.

Schedule a call

References

  1. EU AI Act, Regulation (EU) 2024/1689. Official Journal L 2024/1689, June 2024. EUR-Lex 32024R1689
  2. NIST. Artificial Intelligence Risk Management Framework (AI RMF 1.0). January 2023. doi.org/10.6028/NIST.AI.100-1
  3. ISO/IEC 42001:2023. Information technology : Artificial intelligence : Management system. International Organization for Standardization, 2023.
  4. US Executive Order 14110 on Safe, Secure, and Trustworthy Artificial Intelligence. Federal Register Vol. 88, No. 210, 2023.
  5. Rao, Jaggi, Naidu. "MEDFIT-LLM." IEEE RMKMATE 2025. doi.org/10.1109/RMKMATE64574.2025.11042816
  6. EU AI Act Article 5 (prohibited practices) and Article 99 (penalties). EUR-Lex 32024R1689
← Module 4: Jailbreaks Module 6: Practical Safety →