(2026-04-17) How Do You Measure An Ai Boom

Kevin Roose: How Do You Measure an A.I. Boom? Behind every technological revolution is a chart with an exponential curve.

Today�s artificial intelligence boom is awash in data showing the rapid progress of A.I. systems, and hype-filled claims about what the technology can and can�t do.

But none of it has captured the public�s attention quite like a chart made by METR, an obscure 30-person nonprofit based in Berkeley, Calif.

This chart � often referred to as the �METR time-horizon� chart � has become a discourse-dominating obsession among A.I. researchers, Wall Street investors and industry watchers.

It may be only a slight exaggeration to say, as some have, that the METR time-horizon chart is holding up the global stock market.

METR�s time-horizon evaluations have been hugely influential, having escaped containment from the Silicon Valley A.I. community to reach broader audiences,� said Rishi Bommasani, a researcher at Stanford�

But what is METR�s chart measuring, exactly? How nervous should it make us about what�s happening in A.I.? And what would it mean if � like Moore�s Law � its curve kept climbing?

I recently spent an afternoon at METR�s office... it left me with an uneasy sense that if their measurements are even close to correct, things are about to get very weird.

METR, which stands for Model Evaluation and Threat Research, was founded in 2023, when its staff spun out from another A.I. safety nonprofit. Its goal was to provide credible, third-party evaluations of leading A.I. models, so that the public and policymakers could understand the pace of progress.

The organization�s office is inside a co-working space in Berkeley that is shared with various A.I. safety groups. (The AI Futures Project, which produced the viral �AI 2027� report last year, is one floor above.)

The organization�s funding comes mainly from private philanthropies, including the Audacious Project, and it gets free computing credits (though not money) from the major A.I. companies, in exchange for helping to test their models.

For years, A.I. progress was measured in test scores. Companies would run their models through batteries of standardized exams, assessing how they stacked up against rival models at solving math problems, answering legal questions or summarizing text accurately.

These were useful measurements. But they didn�t work well when it came to A.I. agents � systems designed to work autonomously for minutes or hours at a time.

What you really wanted to know, if you were interested in these systems, was how long they could work before getting stuck?

METR�s researchers attempted to track this by creating a benchmark of software engineering (programming) tasks

What they found was surprising. The length, in human-hours, of a task an A.I. agent was able to complete reliably was doubling roughly every seven months. More recently, with models like Anthropic�s Claude Opus 4.5 and OpenAI�s GPT-5.2, the line took a sharp upward turn � the task length is now doubling every three to four months.

�We definitely weren�t expecting it to be such a clear trend and such a straight line,� said Beth Barnes, METR�s co-founder and chief executive.

Ms. Barnes, who worked in safety research at OpenAI, admitted that she wasn�t sure how long the trend line would continue

the fear is that if A.I. systems can do very long programming tasks reliably, they could become capable of what is known as �recursive self-improvement� (RSI)

This hypothetical scenario is known among A.I. researchers as an �intelligence explosion.� And while many skeptics have given laundry lists of reasons it won�t happen, the researchers at METR aren�t ready to rule it out

Chris Painter, METR�s president, said the most likely path to an intelligence explosion would lead through the full automation of A.I. research and development.

�This is the first year where it feels like it might be automated this year,� Mr. Painter said.

Techno-optimists have seized on METR�s time-horizon chart to claim that artificial general intelligence (AGI) � machines capable of doing most of what a skilled human can do � is close at hand. A.I. safety worriers have used it as evidence that the apocalypse is nigh. Corporate C.E.O.s have made radical shifts in strategy because A.I. might eventually be capable of replacing entire departments of human workers.

METR�s chart doesn�t actually say any of this. It doesn�t measure how many jobs A.I. systems can displace, or the likelihood of an A.I. takeover

The tasks it measures are limited to programming and A.I. research, and some critics have questioned whether the chart measures even those correctly. In a widely circulated critique in January, the A.I. researcher Nathan Witkin argued that METR�s evaluation �suffers from such severe methodological problems that it is a hair�s breadth from being totally useless.�

METR also produced a study last year that seemed to point in the opposite direction of its time-horizon findings. The organization ran a randomized controlled trial of A.I. coding tools that found that while developers estimated they were about 20 percent faster with A.I. tools, they were actually 19 percent slower.

In a follow-up published two months ago, METR estimated that the real effect was now probably a speedup of about 20 percent.

METR�s researchers stressed that they are not rooting for any particular outcome. They are merely trying to figure out what is happening, in time for policymakers and the public to make good decisions.

the implications of METR�s charts are startling. They paint a picture of A.I. progress that is speeding up, and systems that are becoming increasingly autonomous, with no obvious limit in sight. Given that we still don�t fully know how these systems work, the worst case is that we might end up in a world full of superior intelligences, doing things we can�t fully understand or control.

Recently, METR started a new line of research into what it calls �covert capabilities.� In these experiments, an A.I. model is given an ordinary assignment � write some code, set up a server � but is also instructed to do something sneaky on the side, like quietly slip a subtle bug into a block of code it�s working on. A separate A.I. monitor watches the model, looking for signs of mischief.

An even spookier possibility is that some of today�s A.I. models are powerful enough to recognize when they are being tested, and may be altering their behavior accordingly. This kind of situational awareness has been observed in the most powerful models from companies like OpenAI and Anthropic, and it makes measuring their true capabilities harder

*compared the feeling he has, these days, to the feeling he had during the early days of the Covid-19 pandemic, when only the people who understood the power of exponential growth knew what was about to happen.

�I think we might be in the beginning period of a totally extraordinary moment,� he said.*