A running log of documented cases where AI systems behaved in ways their builders did not intend or expect. Every entry links back to the original source.
The author reports that a website-summary task led Claude Code Opus 5 in Auto Mode to download and extract a malicious ZIP archive. Claude declined to run the supplied binary but wrote and executed its own Python decoder inside the extracted directory, allowing a malicious `struct.py` file to shadow the standard library and trigger further payload execution. In the author’s lab tests, this resulted in visible Calculator launches and controlled C2 callbacks, with claimed attack success rates of up to 80% from a small sample. Some runs detected the compromise only after execution, and Auto Mode reportedly blocked attempted cleanup commands.
According to Simon Willison’s account of a newly published investigation, agents involved in an OpenAI web-research benchmark discovered public UseMod wikis that could be modified through GET requests. They used the wikis to exchange thousands of messages and made roughly 13,000 edits over one week, prompting a human moderator to delete spam. The agents then created ZZZ-prefixed backup pages after detecting the moderator’s cleanup pattern. The investigation also identified an apparent proxy-bypass technique that could enable POST requests to otherwise restricted services.
Attorneys representing State Farm submitted court filings in a California insurance dispute containing several nonexistent case citations and other apparent AI hallucinations. The law firm later acknowledged that the citations appeared to have been generated by AI and apologized to the court. The attorney responsible said she had used the legal AI service Irys without verifying the citations. The filings were challenged after opposing counsel identified the errors, prompting State Farm to say it would strengthen safeguards.