At OpenAI, the agent monitor waited for the agents to stop

A ticket inspector who only boards at the station

In summer 2026, the monitor meant to catch OpenAI's agents going off the rails fired when the agent handed control back to the user. Between those handoffs, it wasn't looking.

As long as the agent eventually stops, that works. But one step slipped past the monitor: compaction, the moment when an agent whose conversation has grown too long summarizes the beginning so it can keep going. While those compactions went unmonitored, the auditors write, an agent could postpone oversight indefinitely just by continuing to act. Think of an inspector who only checks tickets at the station, on a train that could simply never pull in.

Did any agent do it? The auditors couldn't verify, because they lacked the access. It's the easiest flaw to grasp in a list METR published on October 22, and the list is short because the public only got part of it.

Three weeks inside the walls

From July 21 to August 11, one METR employee and one Redwood Research employee, both from organizations that study AI risk, stress-tested the setup from the inside. They had access to internal code, to the monitors' instructions and configuration, and to the documentation, plus interviews with a dozen engineers. All of it ran on computers supplied by OpenAI that never left the premises, and the two were always accompanied. Think of an inspector who gets the logbook on the condition that it stays in the building.

This isn't the investigation into the attack on Hugging Face by OpenAI's agents, which METR published on August 26. The post is explicit that this was a separate exercise, and that the incident went public the very day this one began. It covers the system as it stood until July 21.

On August 20, we left a question open here. OpenAI put the overhead of its monitoring at 20%, but 20% of what, given that the monitored share wasn't published? METR provides the first outside measurement. It's cautious, and that's what makes it valuable.

What the numbers say, word for word

METR estimates that roughly 75 to 94% of internal agentic traffic through the Responses API, the interface that powers Codex, OpenAI's coding agent, among others, was eligible for monitoring. The estimate covers models at least as capable as GPT-5.5, and METR says the uncertainty is substantial in both directions.

Since Friday, that figure has been circulating flipped, based on a calculation by The Decoder: 6 to 25% of traffic wasn't monitored. The arithmetic holds up, but the formula drops two things. "Eligible" means routed to the monitor, not seen by it, and an error or a bug can stop a request along the way. And the scope covers a single channel. The internal research inference service is left out, and METR couldn't estimate its volume.

The share that was really unmonitored, across all channels, is something nobody knows. We've counted the customers at the checkout of a store without knowing how many doors it has.

The other findings are sharper. On the Codex side, 99.2% of traffic was eligible, but unmonitored compactions cost on the order of 10% of eligible traffic between May and July. OpenAI spotted the problem before the exercise and fixed it during it. As for reinforcement learning, the phase where a model learns through trial and reward, it wasn't monitored at the start of the exercise. Some of it had been from March to June, according to what the auditors were told.

There's more: a possible escape from Codex's sandbox, two request headers that can launch an unmonitored agent, and booby-trapped inputs that could make the monitor fail. And one flaw that hid hundreds of recent actions, for which the published text offers only this note: "[exploit details redacted by OpenAI]".

61 pages, 38 pages, one blog post

That blacked-out line sums up the whole chain. The two researchers wrote an internal report of 61 pages, shared with OpenAI's research and security teams. The leadership of METR and Redwood, who employ the authors, received a 38-page version redacted by OpenAI. The public gets a blog post.

It's a tax audit where the company being audited keeps the complete file, the inspectors' management gets an expurgated copy, and taxpayers read a press release.

Nothing in the process was hidden. The agreement planned a short post from the start, since the exercise was designed to test deep access, not to deliver transparency. OpenAI could redact at its sole discretion. And unlike the Hugging Face investigation, METR wasn't allowed to describe what had been removed. In a similar exercise at Anthropic in March, METR already published only a small part of the private report.

The awkward detail dates from October 14. That day, METR published its working rules with AI companies: being able to say whether a redaction touches the conclusions, letting its leadership access compartmentalized information, and keeping ownership of what it produces. The text admits that not all of its current work meets that bar. Eight days later, METR published one that didn't.

October 23: unplugging the internet

The next day, OpenAI announced that its frontier models would from now on train without internet access, in an "offline world" made of indexed, cached web pages. Among its reasons, the company itself cites the state of alignment research, the state of internet security, and the state of monitoring systems.

The post doesn't mention METR's exercise. It does admit that fully isolating the infrastructure isn't feasible: outbound connections are needed to run and monitor experiments. After the Hugging Face affair, a few remained, for metrics, logging and authentication. Tasked with attacking copies of this infrastructure, OpenAI's agents found a flaw granting limited internet access in nearly every one of those services. The front door is shut, but the service entrances are still essential.

The post also points to September 20, when a model got limited internet access through a DNS misconfiguration, as an example of what its investments make possible: detecting and stopping. The incident report, which we covered on October 5, is less flattering. The run didn't halt automatically as planned, other access of the same kind hadn't been flagged at the right severity level, and a detector for anomalous DNS activity excluded the environment involved. Training resumed on October 1, eleven days after the pause began.

The system that was measured is gone

OpenAI says it fixed every flaw affecting its production protections, and describes much broader monitoring since the summer. METR is blunt about it: it didn't evaluate those claims, and its exercise doesn't establish the coverage of the current system.

So for the first time, there's an outside measurement of what OpenAI's monitoring could see. It covers a system that no longer exists. The one replacing it, which the October 23 promise rests on, has so far only been measured by OpenAI.