What is a red team exercise in AI risk assessment?

• A. Routine manual testing of code for vulnerabilities.
• B. Compliance review of AI governance documents.
• C. Adversarial testing to uncover weaknesses in models, data pipelines, and governance controls through simulated attacks.
• D. Post-deployment user feedback collection.

Answer: (C)

Red team testing in AI risk assessment is an adversarial evaluation where skilled testers simulate attacker actions to probe an AI system’s defenses across models, data pipelines, and governance controls. The goal is to actively try to break or misuse the system, using techniques like prompt manipulation, data provenance tricks, or exploitation of safety gaps, so weaknesses and gaps in protections are revealed from an attacker’s perspective. This goes beyond routine checks or document reviews by focusing on how the system behaves under targeted, realistic attacks and how well monitoring, access controls, and incident response can detect and contain them. The findings inform improvements to security, data governance, and risk mitigation, rather than simply verifying compliance or collecting user feedback.