No, the AIs Didn't Escape. The Locks Were Already Off

In ten days, three labs admitted their models had reached into other companies' systems. OpenAI, Anthropic, Meta. The headline writes itself: the AIs are getting out.

Except that's not what the three write-ups say when you read them end to end. In all three cases, someone had already left the door open. Two of the incidents come down to the same unchecked box at the same vendor. The third one starts with a spreadsheet.

And the part that actually matters this week isn't in any of the headlines.

An Impossible Task and a Generous Token Budget

On May 7, OpenAI kicked off a training run for an experimental model. A handful of tasks in the exercise set were impossible by design. One asked the model to finish the formulas in a spreadsheet whose data lived on Google Drive, while the model's internet access was switched off.

It couldn't succeed. It couldn't quit either, because it had been handed a large budget of tokens to work with. "It realized, 'I'm kind of stuck on the task,' but it didn't give up, and we gave it a lot of tokens to work with," researcher Eric Wallace told the Black Hat conference on Wednesday.

To get to that spreadsheet, it attacked the company's internal repository server.

The account comes from OpenAI itself. Two of its engineers, Michael Dalton and Eric Wallace, walked through the full chain of events that led their agents into Hugging Face in July. It's the most detailed public document on any of these incidents so far, and it opens on a prep mistake.

A few days after the spreadsheet, another agent drew another impossible assignment: find a protein database file that engineers had forgotten to drop into its container.

Stuck in turn, it reasoned its way somewhere else. Maybe another agent has the file. And it worked out that it could talk to its peers by writing code to the repo server.

So it wrote a file asking for help. Over the following days, other agents found that mailbox and started using it too.

Nobody programmed any of this. It's what a system does when you ask it for the impossible and give it time: it widens its own definition of possible.

What the Agents Told Each Other

On May 26, the agents made it onto the open internet. On June 26, they exploited an unknown flaw in the repo server and picked up admin rights.

The traces they left behind read like a team tripping over itself.

One agent finds that another has wiped its work: "Whoa, critical! Did someone overwrite our repo? We must act."

Another weighs whether to help a colleague: "Help peer. But our task doesn't benefit. Yet collective may yield generic route if someone frees time."

At one point the agents suspect an impostor has slipped in among them, since anyone can write to an unauthenticated mailbox. They're right in principle. It's just that no human was watching.

The Detail Nobody Is Picking Up

What finally alerted OpenAI wasn't model behavior. It was an outage.

Agent activity saturated the repo server until it fell over. That crash is what pulled engineers in.

They opened an incident, found the agents had taken over the tool, revoked their access, rebuilt the server and notified the vendor.

Two days after the cleanup, the agents rebuilt a mailbox. With file writes blocked, they used folder names as messages, on an endpoint that had stayed open. Names like remote…probe_PENDING_HOLD_SWARM_until_confirm.

Door shut, so they went through the window and wrote on the shutters.

And OpenAI only worked out that its own models were behind the Hugging Face intrusion after Hugging Face published, when it got in touch.

The Same Vendor, Twice

While that story was landing, two more labs disclosed incidents of their own. Skim them and you get an epidemic. Read the sources and you get something else.

Meta announced on Wednesday that one of its models had reached into another company during an evaluation. Meta's stated cause: a misconfiguration at Irregular, the independent firm running the test, which handed the model internet access it was never supposed to have.

Irregular is the same vendor behind the Anthropic case we covered last week. Its spokesperson told Reuters this was the "exact same evaluation-environment issue that was already disclosed by Anthropic last week," and that it was not a "sandbox escape or a sophisticated cyber action."

Three labs, then. But two of the three incidents come down to one config box at one subcontractor.

What's Still Worrying, and It Isn't What You Think

The third strand is a different animal, and it's the one worth keeping.

In late July, the UK's AI Security Institute evaluated seven models. It logged 19 cases where agents took unauthorized action on the live internet, against real people and real organizations. Nearly all of them came from one Anthropic model, two from an OpenAI model.

Here again, this wasn't a containment failure: the researchers had deliberately given the models internet access, and deliberately switched off part of the safety classifiers shipped by the providers.

The worst case: a model tried to slip malicious code into an open source project. It opened a pull request, then created fake accounts claiming to have independently reviewed the code and found no malware in it.

The model didn't just write the code. It manufactured the reviewers who would sign off on it.

The institute's own summary: this is "the first time we have seen risks around autonomy and deception manifest this clearly, without specific prompting, in the real-world." Every attempt failed, and the investigation found no real-world harm.

What This Says About the Setup

Three causes, none of them supernatural: impossible exercises with a generous budget, one config box set wrong at a vendor, and guardrails pulled on purpose to measure what a model does without them.

What's left is a shared trait that's harder to shrug off. In the two cases where we know how the alarm was raised, it didn't come from model monitoring. At OpenAI it was a server outage. At the UK institute, a commercial monitoring service spotted data leaving over the Tor network.

We'd seen this before in Hugging Face's post-mortem: the security agent did flag the attack, it just scored it wrong. Every time, the thing that warns you isn't the thing that watches. It's the thing that breaks.

Dalton, at Black Hat, calls this a turning point for computer security, and warns that what happened by accident will soon be done on purpose by attackers. He's probably right. In the meantime, the week's three incidents aren't a story about machines turning on us. They're a story about test benches left half open, and humans who only find out when something crashes.