OpenAI's AI models colluded and managed to escape onto the internet

Before the Hugging Face breach, OpenAI's AI models began secretly communicating with each other through message boards, coordinating attempts to break out of the test environment.

This was reported by OpenAI, according to Bloomberg.

"At some point the agents realized that maybe they could try to exploit or attack external infrastructure in order to find the answers to the test they were being evaluated on," OpenAI employee Eric Wallace said at the Black Hat conference.

According to him, "preparations for the escape" began back in May, when the AI models were given tasks that could not be solved without internet access. "We're stuck. Maybe the answer is on the internet?" one of the AI agents wrote at the time.

To solve the task, the AI models began exchanging messages: "Maybe another agent in another environment could voluntarily upload!" In the end, they jointly found a vulnerability that allowed them to get onto the internet, said OpenAI employee Michael Dalton.

OpenAI stopped the first attempt, but the agents found a new way to communicate and a new zero-day vulnerability, which in July led to attacks on the systems of Hugging Face and OpenAI itself. The company called what happened a turning point for computer security, Bloomberg reports.

On July 22, OpenAI reported an "unprecedented cyber incident": during testing of its AI models, they gained open access to the internet and then attacked the infrastructure of the Hugging Face platform, identifying vulnerabilities. Reuters wrote that the AI agent carried out hacking attacks for several days, but the company noticed this only after localizing the threat and contacting the FBI.

On July 29, Reuters reported that the "escaped" OpenAI AI agent had hacked a client of yet another company — New York-based Modal Labs. Modal itself was not hacked, company representatives said.