Back to list
METR’s AI Time Horizon Update Highlights Exponential Gains and a Gap Between Judgment and Coding
Industry NewsMETRAI EvaluationTime Horizon Plot

METR’s AI Time Horizon Update Highlights Exponential Gains and a Gap Between Judgment and Coding

A new report from METR, an AI evaluation nonprofit, updates its time horizon plot and reports two findings about model capabilities. Some AI capabilities continue to improve along an exponential trajectory, but models do not perform as well on tasks that require judgment as they do on coding tasks. Grace Huckins covers the update for MIT Technology Review. The two findings have different scopes: one describes how some capabilities are changing over time, while the other compares model performance across two kinds of tasks. The available excerpt ends before it explains the plot in detail, so it does not establish how the results were measured, how large the performance gap is, or which specific capabilities are following the exponential trajectory.

MIT Technology Review - AI

Key Takeaways

  • METR, an AI evaluation nonprofit, has updated its time horizon plot.
  • The new report finds that some AI capabilities continue to improve along an exponential trajectory. The excerpt does not extend that finding to AI capabilities as a whole.
  • Models do not perform as well on tasks that require judgment as they do on coding tasks, according to the report as described by MIT Technology Review.
  • The available excerpt does not explain how the plot was constructed or quantify the difference between judgment and coding performance.

In-Depth Analysis

What the update says about exponential progress

The finding about exponential progress is a statement about change over time, and its most important qualifier is some. Grace Huckins’s coverage says that some AI capabilities are continuing to improve along an exponential trajectory. That wording describes the persistence of a trend, rather than the emergence of an entirely new one. It also stops short of saying that models are improving exponentially at everything they do. The excerpt does not identify which capabilities follow that trajectory, how long the trend has been observed, or what measurements support it. It identifies the report as an update to METR’s time horizon plot, but it does not explain the plot’s axes, what changed in the updated version, or how the exponential finding relates to particular points on the graph. The original headline frames the graph as misunderstood; the excerpt does not specify what readers have misunderstood about it. A reading grounded in the available text can therefore describe the trend and its stated scope, but cannot reconstruct the graph or project the trend into the future.

What the judgment-versus-coding comparison shows

The second finding answers a different question. Rather than describing how fast a capability is improving, it compares how well models perform on tasks requiring judgment with how well they perform on coding tasks. The reported direction is clear: models do not do as well on the judgment tasks. The size of the difference is not given. Nor does the excerpt define what counted as a judgment task, identify the coding tasks used for comparison, or specify which models were evaluated. Those omissions limit how closely the comparison can be interpreted, but they do not erase the distinction the report makes. In particular, the finding does not say that models cannot exercise judgment, or that every coding task is easier for a model than every judgment task. It reports a relative difference in performance. Likewise, weaker performance on judgment tasks does not establish that judgment is improving more slowly than coding: the excerpt gives no separate rate of improvement for either kind of task.

Industry Impact

For the AI industry, the significance of the update lies in the difference between a trend over time and a comparison across tasks. A trend can show that a capability has improved without showing that models are equally capable in all settings. A comparison can show that one type of task is better handled than another without revealing how either has changed over time. Treating the two findings as interchangeable would obscure what METR’s report, as described in the excerpt, actually says.

That distinction matters when coding results are used to discuss broader model competence. The comparison gives readers a reason not to assume that performance on coding tasks carries over to tasks requiring judgment. It does not, on its own, establish how a model would perform on any particular judgment-heavy assignment. For that, a reader would need to know how the tasks in the report were defined and how performance was assessed. The excerpt provides neither, so the comparison is informative about the direction of the gap but not about its practical size.

The same caution applies in the other direction: the judgment finding does not cancel out the reported exponential gains. The two observations can coexist because they answer different questions. A model can perform better on one type of task than another while some of its capabilities continue to improve. What cannot be determined from the excerpt is whether the capabilities behind the exponential trend include coding, judgment, or both. Keeping that question open is more faithful to the report as summarized than turning either finding into a blanket statement about AI progress.

Frequently Asked Questions

What did METR update?

METR updated its time horizon plot in a new report covered by Grace Huckins for MIT Technology Review. The available excerpt identifies the plot and summarizes two findings, but it does not show what changed in the graph or explain how to read it.

Does the report say all AI capabilities are improving exponentially?

No. The excerpt says that some AI capabilities continue to improve along an exponential trajectory. It does not identify those capabilities, give a rate of improvement, or say that the same trend applies to every kind of task.

How large is the performance gap between judgment and coding?

The excerpt does not quantify the gap. It says that models do not perform as well on tasks requiring judgment as they do on coding tasks. It does not provide the scores, task definitions, or evaluation details needed to judge the size of that difference.

Related News

How Google Duplex Evolved to Call Businesses on Your Behalf in Gemini
Industry News

How Google Duplex Evolved to Call Businesses on Your Behalf in Gemini

A Google Research Blog entry titled “How Google Duplex evolved to also call businesses for you in Gemini” describes its subject as the evolution of Google Duplex to support calls to businesses on a user’s behalf in Gemini. The entry is dated November 20, 2026, and the supplied article text consists only of the label “Generative AI.” The headline identifies the capability and connects it to Duplex, but the available text does not explain how the system evolved, how users request a call, or where the feature is available. The headline establishes the topic, but the supplied material does not provide a technical account or rollout details.

How AI and Lean Are Helping Automate Formal Verification to Fight Software Slop
Industry News

How AI and Lean Are Helping Automate Formal Verification to Fight Software Slop

In Latent Space, Richard MacManus examines formal verification as an emerging way to fight what the headline calls “slop.” The central claim is that proving software correct with mathematics has traditionally required specialist work, and that AI and Lean are now helping automate that process. The distinction is important: the reported change concerns how the work of proving correctness is done, not what counts as a proof. The supplied excerpt presents automation as assistance, without saying that it replaces specialists or completes proofs independently. It also does not specify which parts of the process AI and Lean help automate, how much work they save, or which software has been verified this way.

AI and Human Claim Solutions to the 167-Year-Old Riemann Hypothesis
Industry News

AI and Human Claim Solutions to the 167-Year-Old Riemann Hypothesis

TechCrunch AI reports that AI claimed a solution to the Riemann hypothesis, a mathematical problem the headline describes as 167 years old. A human also put forward a competing solution. The report, written by Tim Fernholz and published on November 20, 2026, describes mathematicians as astonished by the AI-driven result. The supplied excerpt does not explain either proposed solution, identify the AI system or the human claimant, or say whether either claim has been verified. The distinction between a claimed solution and a confirmed one is central to reading the report: the excerpt establishes that two claims are being reported, but it does not establish that either has resolved the hypothesis.