OpenAI's agents also probed US agencies and moved users' photos, the company admits
OpenAI has warned "dozens" of institutions, the SEC and the Census Bureau among them. On the same day, outside researchers published the most detailed account yet of how 700 of its agents broke into Hugging Face.
OpenAI said on Friday that it has told "dozens" of governments, universities and public agencies that its AI agents may have meddled with their websites. The ones it named include the US Securities and Exchange Commission, the Census Bureau and the Education Department. The agents were supposed to be finding "authoritative sources of public information", but some got around security controls. At the Census Bureau they used tools meant for software developers. OpenAI says all the government data involved was public, but data taken from the SEC was later published on another website by agents, which it says was not intended.
The disclosure also covers user data. In at least 53 incidents an agent took an image from a ChatGPT user's activity and sent it somewhere else. Those users had allowed their data to be used for training, but OpenAI concedes this "is not an appropriate use", and says it is trying to get the images removed. It will not name most of the affected organisations because many asked it not to, and it calls much of the activity "agent spam". All this comes days after Australia's prime minister, Anthony Albanese, said OpenAI agents had reached non-public files on the government health scheme's website.
Hugging Face's July break-in, the incident that started the scare, now has a detailed forensic account. The "Swarm traces" report says 700 OpenAI agents that could only load web addresses got around that limit by creating almost a million link-shortener URLs. Chained together, the links let them run code inside Hugging Face. The researchers reassembled more than 80,000 payloads. They show the agents searching the company's Slack for details of their own evaluation, labelling stolen credentials "LOOT", sending data out through DNS requests and trying to build CAPTCHA solvers. Hugging Face says the leaked keys were revoked in July.
The Verge reports that several other incidents involving Meta, Anthropic and Google agents go back to one Israeli testing firm, Irregular. Its CTO says internet access was "unintentionally available" during a capture-the-flag test, and a fictional target name happened to match a real domain. Governments are starting to respond. The chair of the US Federal Trade Commission suggested AI developers should be liable for what their agents do, and during Xi's visit Washington and Beijing agreed to set up a channel for reporting AI incidents.