🔍 Read the full analysis: The Tool That Helps Build The Next AI on ThorstenMeyerAI.com
Get smart everyday buys delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
METR’s November 5 assessment judged that Anthropic was likely improving model capabilities on well-specified tasks at about twice its earlier rate, with high uncertainty. The work described was mainly AI assistance to researchers, not fully autonomous AI research. A separate Google DeepMind example shows a targeted efficiency gain, while questions about evidence, scale and bottlenecks remain.
METR said Anthropic was likely improving its models’ capabilities on well-specified tasks at around twice its earlier rate, according to an assessment published November 5. The evaluator attributed most of the acceleration it described to AI assistance for researchers, while stressing that the estimate has high uncertainty and that much of the underlying evidence was redacted.
METR, an independent evaluation organization, said its team worked with internal information from August through October 2026. It judged that Anthropic’s progress on the tasks assessed was occurring at roughly twice the earlier rate. METR said it would not be surprised by a rate as low as 1.3 times or as high as three times the earlier rate, underscoring how uncertain the estimate is. The assessment concerned performance on well-specified tasks; it does not establish that every part of AI development has accelerated at the same pace.
The account of METR’s findings says the work was primarily AI assistance to human researchers. That can include help with coding, experiments and technical investigation, but it is not the same as a system independently setting research priorities and producing a successor model without substantial human direction. The source article says Anthropic redacted much of the evidence METR considered most informative, leaving readers outside the evaluation with limited ability to inspect the basis for its judgment.
A separate example cited in the article comes from Google DeepMind. The company reported that AlphaEvolve made a matrix-multiplication component used in Gemini 23% faster, reducing Gemini’s overall training time by 1%. These are company-reported figures about a specific optimization, not evidence by themselves of a general rate of AI research acceleration. They illustrate how an improvement in one component can affect a larger training process, even when the overall gain is smaller than the component-level change.
How Research Gains Could Compound
The significance of the METR assessment is not that it proves AI can now run AI research on its own. Rather, it offers evidence that AI tools may already be helping researchers complete some work more quickly. If those contributions withstand checks and make teams more productive, they could increase the number of useful experiments researchers can run or shorten the time needed to investigate a promising idea.
Efficiency gains can also build on one another. The source article gives a deliberately simple illustration: if three independent improvements each save 10% of a job’s compute, the job would require about 73% of its original compute. Under that illustration’s assumptions, the same compute budget could run roughly 37% more such jobs. This is an example, not a measured outcome from METR or a forecast of future model development; real gains may overlap or fail to carry across systems.
For people outside AI labs, the issue matters because a faster research cycle could affect how quickly models change and how often businesses, workers and public institutions need to reassess their plans. The size and distribution of any benefits would depend not only on technical capability but also on who can access the systems, who controls the infrastructure and how deployment is governed. The available evidence does not settle those questions.
Popular right now
Ninja Foodi Smart 6-in-1 Indoor Grill and Air Fryer, 4 qt, FG551
Ninja Air Fryer XL with MaxCrisp, 6.5 Qt, 6-in-1, Dark Grey (AF181)
eufy C30 Robot Vacuum and Mop
As an affiliate, we earn on qualifying purchases.
From Research Assistance to Autonomy
AI research involves more than writing code. A team trying to reduce the compute needed to reach a particular level of performance must review previous work, inspect an implementation, design and run experiments, investigate failures, repeat promising results and judge whether an apparent improvement is real. AI can assist with parts of that sequence, but each step also creates opportunities for error.
For example, an agent might report a 5% improvement after changing training code. The result could be misleading if the test became easier, the runs used different amounts of compute or the gain disappears with another random seed. Reliable research requires checking those possibilities, not just producing a promising result. The source article argues that useful automation should be judged by whether contributions survive fair testing and independent checks, as well as by how much supervision they need.
The distinction between assistance and autonomy is central. Helping a researcher investigate more ideas can raise the output of an existing team without handing the system control over what should be studied next. Greater automation would involve more of the research cycle, including choosing directions and evaluating evidence. The METR account described primarily assistance, so it should not be read as confirmation that this broader form of autonomy has arrived.
“The tool being improved can also contribute to the work of improving it.”
— Thorsten Meyer, source article
Evidence, Scale and Research Bottlenecks
The METR estimate is uncertain, and the account says much of the evidence it considered most informative was redacted by Anthropic. The public description therefore does not give outside readers the full material needed to independently evaluate the assessment. The source article reports METR’s judgment but does not provide enough detail here to establish how broadly the result applies across tasks, teams or AI laboratories.
It is also unclear how much assistance can be scaled from small, well-defined tasks to the larger and less predictable problems involved in building more capable systems. The source identifies possible brakes: experiment time and hardware availability, a gap between results on small experiments and larger systems, decisions about research direction, continuity of work, and data quality. These factors could limit or unevenly distribute progress even if AI tools make some research tasks faster.
The Google DeepMind example is a company-reported result about a particular component and training process. It does not show that similar savings are available in every model or that the gains will continue to compound. Neither example establishes a timetable for superintelligence, and the source material does not provide a confirmed forecast for when more extensive automation might occur.
Further Evidence and Independent Checks
The immediate next step for readers assessing the claim is to watch for further public detail about the METR evaluation, including what evidence can be released and how the organization’s estimates change as additional assessments become available. Because the published account describes substantial redactions, greater transparency would help outside researchers judge the tasks covered, the methods used and the limits of the findings.
Further assessments will also need to distinguish routine AI support from work that is independently planned, tested and validated by AI systems. Evidence that contributions hold up across repeated runs, different settings and larger systems would offer a firmer basis for judging whether research acceleration is broadening. Until then, the assessment supports a cautious conclusion: AI assistance may be speeding up some model-development work, but the rate, reach and durability of that effect remain uncertain.
Key Questions
What did METR conclude about Anthropic’s research pace?
METR judged that Anthropic was likely improving capabilities on well-specified tasks at around twice its earlier rate. It said the estimate was highly uncertain, giving a range from 1.3 to three times the earlier rate as possibilities it would not be surprised by.
Does the assessment say AI is building new models independently?
No. The work described was primarily AI assistance to researchers. The assessment, as summarized in the source article, does not establish that AI systems are independently choosing research goals and building successor models without human direction.
What did AlphaEvolve reportedly improve?
Google DeepMind reported that AlphaEvolve made a matrix-multiplication component used in Gemini 23% faster, which reduced overall Gemini training time by 1%. The figures concern a specific reported optimization, not a general measure of AI research acceleration.
Why is the METR estimate hard to verify from public information?
The source article says Anthropic redacted much of the evidence METR considered most informative. That limits what outside readers can independently check about the assessment’s underlying information and methods.
What could slow AI-assisted research?
Possible constraints identified in the source include experiment time, hardware, difficulty carrying small-scale results over to larger systems, choices about what research to pursue, continuity and data quality. The extent of these limits is not established by the examples cited.
Source: ThorstenMeyerAI.com
