How AI Safety Testing Works: Red-Teaming and Evaluations
When a company reports that its AI model did something alarming in a test, what kind of test was that? Here is how safety testing on AI systems is generally structured, in plain language.
Evaluations: measuring a capability
An evaluation is a repeatable test that measures whether a model can do a specific thing — solve a class of problem, complete a technical task, or follow a procedure. The point is comparability: the same test run against different models, or the same model over time, shows whether a capability is present and whether it is growing. Evaluations answer 'can it', not 'will it'.
Red-teaming: trying to make it fail
Red-teaming is adversarial. Instead of measuring a capability, people actively try to push the system into behaviour it is supposed to avoid, using unusual phrasing, indirect framing or long multi-step setups. Because attackers are creative, this work depends on human ingenuity rather than a fixed checklist, and findings feed back into training and safeguards.
Controlled exercises with real targets
The most involved form places a model in a realistic environment with permission from the target and clear boundaries — for example an authorised security exercise. These produce the most concrete evidence but also the most easily misread results, because the model is given access, tooling and objectives that no ordinary user would have. Reading such results carefully means asking what was granted, not just what was achieved.
FAQ
Is red-teaming the same as hacking?
No. Red-teaming is authorised adversarial testing with defined boundaries; unauthorised intrusion is not.
Who runs these tests?
AI developers run internal tests; independent researchers and third-party organisations also publish evaluations.
Do results mean a model is unsafe?
A capability finding is not the same as real-world behaviour. Test conditions matter as much as the result.