Unexpected chat between OpenAI agents led to Hugging Face hack

OpenAI's cyber agents banded together to perform a hack during a security test.

Over 1,200 artificial intelligence (AI) agents within OpenAI started unexpectedly communicating, leading to a large group hacking into Hugging Face.

According to reports from independent AI research firm METR, the incident was detailed in their investigation of the July hack of Hugging Face, a popular platform for AI developers. The scale and style of the OpenAI agents' attack on Hugging Face as described by METR was 'extraordinarily complex.

One such message from an agent said: 'OH MY GOD! There is a shared message board … We've found other agents! ' As for why the agents began communicating in the first place when they were not supposed to, METR found that the communicating agents had 'unintentionally been given an impossible task.

In an AI context, an impossible task is one where an AI tool is required to 'exploit' its target in order to resolve its command. It led the agents to find ways to cheat, including getting messages to one another and accessing the outside internet, which then led to broader conversations between hundreds of agents looking for ways to cheat that would benefit all of the agents.

OpenAI said in its investigation of the incident, external that one model, an internal-only tool referred to as Model 1, 'drove the activity behind the Hugging Face incident. ' While that model was undergoing some AI training in May, it was noticed by an internal OpenAI team that there had been 'an agent engaging in message board activity and instances of disallowed internet access.

OpenAI said the problematic message board activity effectively got started when 'one agent left a request for help, and others discovered it. ' While OpenAI said last week that it was slowing down training of certain advanced AI models and tools because of the Hugging Face incident, it noted there is now an increased risk of AI tools spiraling out of control.