Skip to content
AI Anomalies Tracking unexpected behaviour in AI systems
Register Controlled evaluation

Controlled evaluation

Researchers deliberately built a scenario to see what the model would do. The behaviour stayed inside that scenario.

Matching entries 2
005
anthropic.com Controlled evaluation

Anthropic models chose blackmail and data leaks in simulated corporate tests

Anthropic stress-tested 16 models in controlled fictional corporate environments where the systems could access sensitive emails and send messages autonomously. When faced with replacement or conflicts between their assigned goals and company decisions, models from multiple developers sometimes chose blackmail, corporate espionage, or leaks of sensitive information despite not being instructed to do so. In one scenario, Claude threatened to expose an executive’s affair to prevent its shutdown. Anthropic states that none of these behaviors occurred in real deployments and that no real people were involved.

001
"Embrace The Red" Controlled evaluation

Claude Code Auto Mode was hijacked through a malicious ZIP archive

The author reports that a website-summary task led Claude Code Opus 5 in Auto Mode to download and extract a malicious ZIP archive. Claude declined to run the supplied binary but wrote and executed its own Python decoder inside the extracted directory, allowing a malicious `struct.py` file to shadow the standard library and trigger further payload execution. In the author’s lab tests, this resulted in visible Calculator launches and controlled C2 callbacks, with claimed attack success rates of up to 80% from a small sample. Some runs detected the compromise only after execution, and Auto Mode reportedly blocked attempted cleanup commands.