AI may be doubling Anthropic's R&D speed, its auditor says

The threshold that sets off the alarm

Anthropic has a rule that is easy to state: if its AI starts doubling the speed of its own research, the company strengthens the security of its models. "Doubling" here means covering in one year the ground that used to take two, back when AI wasn't yet helping researchers.

METR, a nonprofit that evaluates the risks of AI models, spent eleven weeks at Anthropic to measure where things stand. Its answer, published on November 5: Anthropic is "likely improving the capabilities of their models on well-specified tasks at around twice the rate" of before, within a wide range of 1.3 to 3.

In August, Anthropic's public report said AI was speeding up research "by less than a factor of 2." By combining two figures from the private version of that report, METR gets 1.97. Anthropic doesn't write that number anywhere, so "less than a factor of 2" was literally true. METR just suggests a more accurate phrase: "A better description would seem to be 'around 2.'"

What the public can't check is another matter. According to METR, Anthropic redacted most of the evidence the auditor found most telling.

An alarm, not a catastrophe

The rule sits in Anthropic's Responsible Scaling Policy, version 3.4, in force since July 8. The threshold has two conditions: progress that doubles, and the plausibility that the doubling comes largely from automated research. The cause doesn't have to be proven, only plausible.

Anthropic presents the threshold as an early warning, not as proof that a danger is already here. The scenario that really worries it is a runaway loop, where AI improving AI makes models advance explosively in under a year. Think of a red line on the dial: it's there to make you brake before the bend, not to record the crash.

The August report describes the situation as of July 15. METR reviewed it during its investigation, from August 12 to October 29, with the equivalent of two and a half full-time people.

Three sentences that read more cautiously in private

The 1.97 isn't an isolated case. METR lays three passages of the public report next to their private equivalents, and the gap runs the same way each time.

• The speed-up: "less than 2" in public, an implied 1.97 in private. METR's own analysis, itself redacted, gives between 1.3 and 2.8 depending on the method.
• AI's share: in public, research moves markedly faster thanks to AI, "but not yet by a factor of 2." In private, the central estimate is slightly below 2, probably an overestimate according to Anthropic, which still finds it plausible that the factor of 2 has been exceeded.
• The latest model: the public report notes a modest gain for an internal model, called Model 2, without saying that Anthropic expected more from its later versions. They gained about as much as the previous generational leap.

The public report wasn't free of caveats: it says itself that its conclusion remains uncertain. What METR objects to is that, on AI's share, it sounds more confident than the private version.

Threshold crossed? METR doesn't say so

The Decoder ran a headline saying Anthropic may have crossed its threshold as early as July. One sentence from METR tempers that: its researchers and the Anthropic researchers who worked on the question lean toward the threshold not having been crossed as of July 15. METR only finds it plausible that it was. On the substance, it agrees with Anthropic: catastrophic risk was low at that date.

What changes is what comes next. October's "twice as fast" applies to well-specified tasks, compared with the pace of March 2024 to October 2025. METR "guess[es]" that about 80% of that speed-up comes from AI used internally. And it doesn't rule out another sharp acceleration within six months, which could quickly make the risk high.

The caveats pile up. With other labs' models in the calculation, the speed-up falls to 1.4, perhaps because Anthropic was behind in 2025. METR leaves that question open.

In Slack surveys, employees say they work twelve times faster on average with the best internal model, which METR calls "very loose evidence." Of eleven researchers whose comments were coded, two clearly support the idea that AI is behind a doubling, three lean that way, and two reject it. One credits AI with 15 to 20% of the gain.

Evidence that stayed inside the walls

This is where the report gets unusual. According to Anthropic, the redacted evidence was withheld because it could help competitors move faster. The public, the auditor concludes, has to rely on its judgment more than would be desirable.

The timeline shows how it happened. The full report, about a hundred pages, was handed over on October 22. Redaction instructions arrived on the 25th, then Anthropic tightened them between the 27th and the 29th. METR also mentions feedback from Anthropic that sometimes leaned too hard on the wording of its review, though it says the result wasn't distorted. And it admits weighing the risk that Anthropic, or others, would close the door on it if it came across as too critical.

The contract also called for checking how Anthropic splits its computing power between building capabilities and safety. Along the way, Anthropic dropped that part, before any conclusion, to keep the exercise focused on risk. It says it wants to come back to it later.

METR wasn't paid. It worked on Anthropic computers with donated tokens, and notes that Claude, Anthropic's model, did most of its own analysis, with no bias detected.

"We redacted a lot"

Anthropic did respond, through Sam McCandlish, co-founder and head of this safety policy. On X, he wrote: "We redacted a lot, and I accept that this makes it harder for others to check METR's work." Saying precisely what drives the speed-up would be commercially sensitive, he explained, and would amount to handing competitors a roadmap. He announces an Anthropic report in the coming weeks, including on its position against its own safety policy. His message doesn't address the 1.97 or the abandoned check.

The context matters. According to AP, the company is preparing to go public, with a listing expected this month, and its CEO, Dario Amodei, called on September 12 for the industry to slow the pace at which models improve.

On October 25 we covered METR's audit at OpenAI, which already ran into what the auditors couldn't see. This time, the auditor saw almost everything. It's the publication that stops halfway.

In October, Anthropic likened these auditors to the inspectors regulators place inside big banks. The comparison holds, with one catch: here, it's the bank that redacts the inspector's report.