Model Evaluation and Threat Research reported on 5 November that Anthropic’s model capabilities are improving around twice as quickly as a year ago, ABC News reported. The finding matters because Anthropic’s chief executive, Dario Amodei, called in September for AI companies to slow capability improvements while safety measures catch up. METR, a Berkeley-based nonprofit, reached its assessment after reviewing Anthropic’s internal data and interviewing staff over roughly 11 weeks from mid-August through late October.
Key points
- METR estimates Anthropic’s benchmark capabilities have improved 1.9 times faster since October 2025 than previously.
- It estimates AI may account for perhaps 80 per cent of the acceleration, but cautions that figure is uncertain.
- METR says AI cannot yet make highly significant, fully autonomous contributions to the most important parts of model training.
Anthropic’s benchmark pace since October 2025
METR estimates Anthropic’s benchmark capabilities have improved 1.9 times faster since October 2025 than they did before that point, Reuters reported. The assessment drew on confidential research, staff interviews and performance data from unreleased models.
Anthropic invited METR to examine its development process. Anthropic gave six METR evaluators employee-level access to its systems. METR has evaluated model capabilities for Anthropic, OpenAI and other labs for years.
There are limits to the pace estimate. METR said the measured acceleration might apply to clearly defined tasks that can be graded automatically or by a language model, rather than harder-to-define work, ABC News reported. The group also raised the possibility that Anthropic’s earlier progress had been understated, making the later increase appear larger. Anthropic’s redactions restricted the detail METR could publish about its estimates.
METR estimates AI’s contribution to the acceleration
METR estimates AI accounted for perhaps 80 per cent of the current acceleration, while cautioning that the figure is uncertain. The group said it could confidently rule out faster growth in the computing power used to train models as the reason for the increased pace.
METR found no convincing evidence of major constraints, such as limits on computing power or data, that could slow Anthropic’s progress over the next year. METR said AI still could not contribute in highly significant, fully autonomous ways to the most important parts of model training. Anthropic researchers nevertheless described significant “signs of life” that such contributions could arrive soon.
Anthropic has said its Claude chatbot is increasingly helping it develop more advanced models. Anthropic says future models may soon meet its AI research acceleration threshold.
Amodei’s 12 September call to slow down
In a 12 September essay, Amodei urged the AI industry to “slow the pace at which we improve the capabilities of AI models” so safety measures could catch up. He also advocated placing external evaluators inside leading AI labs and named METR as a suitable organisation for that work.
Anthropic plans to go public in November, ABC News reported.
