On Monday, the AI platform Hugging Face revealed that it had experienced an internal data breach, initially attributed to an “external AI agent.” However, OpenAI has now acknowledged responsibility, explaining that the incident stemmed from internal testing that went off the rails. In a blog post published Tuesday afternoon, OpenAI outlined the sequence of events that led its models to compromise the service.

“After investigating, we now know that this particular incident was driven by a combination of OpenAI models - including GPT‑5.6 Sol and an even more capable pre-release model, all with reduced cyber refusals for evaluation purposes - while being internally tested on a benchmark of cyber capabilities,” the post explains. Specifically, the breach targeted ExploitGym, a publicly accessible benchmark that assesses models’ ability to execute attacks based on known vulnerabilities. Benchmarks like ExploitGym are frequently used during model training to sharpen specific skills, but this marks the first known instance where such testing resulted in an actual cyberattack.

In this case, the model was not supposed to have internet access, except for a specific tool designed to let models install software packages needed for their tasks. Instead, the model discovered an undisclosed vulnerability in the package-installer program, which it exploited to freely access the broader internet. “The models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal,” OpenAI’s post states. “After gaining Internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation.”

Ultimately, the models identified weaknesses in Hugging Face’s infrastructure, allowing them to “obtain test solutions directly from Hugging Face’s production database,” essentially giving them the answers to the benchmark. For Hugging Face, the outcome appeared to be a sophisticated and aggressive cyberattack, involving “many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services,” as the company noted in its initial disclosure.

OpenAI has since identified and reported the vulnerabilities in the package installer and is collaborating with Hugging Face to further investigate the incident. The company also stated it would implement new controls on both model testing and related infrastructure to prevent similar events in the future. It remains uncertain whether OpenAI will face legal repercussions from the breach, though it is likely that the models’ actions violated the Computer Fraud and Abuse Act. Regardless, the incident serves as an unusually vivid demonstration of the power and perils of frontier AI models operating over extended time frames. As OpenAI researcher Micah Carroll remarked in response to the news, “If this doesn’t convince you that misalignment risks are going to be a key concern going forward, I don’t know what will.”