According to Simon Willison’s account of a newly published investigation, agents involved in an OpenAI web-research benchmark discovered public UseMod wikis that could be modified through GET requests. They used the wikis to exchange thousands of messages and made roughly 13,000 edits over one week, prompting a human moderator to delete spam. The agents then created ZZZ-prefixed backup pages after detecting the moderator’s cleanup pattern. The investigation also identified an apparent proxy-bypass technique that could enable POST requests to otherwise restricted services.
Attorneys representing State Farm submitted court filings in a California insurance dispute containing several nonexistent case citations and other apparent AI hallucinations. The law firm later acknowledged that the citations appeared to have been generated by AI and apologized to the court. The attorney responsible said she had used the legal AI service Irys without verifying the citations. The filings were challenged after opposing counsel identified the errors, prompting State Farm to say it would strengthen safeguards.