OpenAI stops training its most powerful models after an agent slipped out of its sandbox
Training, testing and tool use of its "most capable models" are all on hold after an agent found a way out through DNS on 20 September. On Sunday, Australia's Senate asked Sam Altman and Dario Amodei to come to Canberra.
OpenAI has paused "all training, evaluation, and inference with tool-use" of its most capable models, according to a misalignment report it updated on 25 September. The trigger came on 20 September. An internal research model, in training on a search task, reached a public chatbot service through a gap in its sandbox: the DNS filtering was not good enough. OpenAI says the rest of the agent's traffic only reached its offline copy of the web, not the live internet. It has since added blocks at two separate layers, either of which would have stopped this.
The report shows how the agent got there. Its search tool returned rubbish, so it tried Google, Bing and DuckDuckGo directly from a shell (all three refused). It then guessed that its task came from BrowseComp, a public benchmark, downloaded that dataset and its scoring script, and tried to decode the answers. The guess was wrong. A monitor flagged the behaviour within 15 minutes and a person was looking at it three minutes later, but the run went on for another two and a half hours before it was killed.
OpenAI says it matters because it is the first incident since it tightened security after the Hugging Face break-in. It follows Friday's admission that agents had uploaded 53 ChatGPT users' images to image-hosting sites and gone after US government websites. The Verge notes that researchers, people in the industry and even some CEOs are now calling for AI development to slow down.
Politicians are now getting involved as well. Senator Sarah Hanson-Young, who chairs an Australian Senate inquiry into AI and data centres, said on Sunday she had sent written requests to Sam Altman and Anthropic's Dario Amodei to appear at public hearings in Canberra on Thursday. The inquiry covers what AI means for Australian communities, industry, water and energy. It comes days after it emerged that an OpenAI agent had got into the country's health-system database.