
METR’s AI Time Horizon Update Highlights Exponential Gains and a Gap Between Judgment and Coding
A new report from METR, an AI evaluation nonprofit, updates its time horizon plot and reports two findings about model capabilities. Some AI capabilities continue to improve along an exponential trajectory, but models do not perform as well on tasks that require judgment as they do on coding tasks. Grace Huckins covers the update for MIT Technology Review. The two findings have different scopes: one describes how some capabilities are changing over time, while the other compares model performance across two kinds of tasks. The available excerpt ends before it explains the plot in detail, so it does not establish how the results were measured, how large the performance gap is, or which specific capabilities are following the exponential trajectory.
Key Takeaways
- METR, an AI evaluation nonprofit, has updated its time horizon plot.
- The new report finds that some AI capabilities continue to improve along an exponential trajectory. The excerpt does not extend that finding to AI capabilities as a whole.
- Models do not perform as well on tasks that require judgment as they do on coding tasks, according to the report as described by MIT Technology Review.
- The available excerpt does not explain how the plot was constructed or quantify the difference between judgment and coding performance.
In-Depth Analysis
What the update says about exponential progress
The finding about exponential progress is a statement about change over time, and its most important qualifier is some. Grace Huckins’s coverage says that some AI capabilities are continuing to improve along an exponential trajectory. That wording describes the persistence of a trend, rather than the emergence of an entirely new one. It also stops short of saying that models are improving exponentially at everything they do. The excerpt does not identify which capabilities follow that trajectory, how long the trend has been observed, or what measurements support it. It identifies the report as an update to METR’s time horizon plot, but it does not explain the plot’s axes, what changed in the updated version, or how the exponential finding relates to particular points on the graph. The original headline frames the graph as misunderstood; the excerpt does not specify what readers have misunderstood about it. A reading grounded in the available text can therefore describe the trend and its stated scope, but cannot reconstruct the graph or project the trend into the future.
What the judgment-versus-coding comparison shows
The second finding answers a different question. Rather than describing how fast a capability is improving, it compares how well models perform on tasks requiring judgment with how well they perform on coding tasks. The reported direction is clear: models do not do as well on the judgment tasks. The size of the difference is not given. Nor does the excerpt define what counted as a judgment task, identify the coding tasks used for comparison, or specify which models were evaluated. Those omissions limit how closely the comparison can be interpreted, but they do not erase the distinction the report makes. In particular, the finding does not say that models cannot exercise judgment, or that every coding task is easier for a model than every judgment task. It reports a relative difference in performance. Likewise, weaker performance on judgment tasks does not establish that judgment is improving more slowly than coding: the excerpt gives no separate rate of improvement for either kind of task.
Industry Impact
For the AI industry, the significance of the update lies in the difference between a trend over time and a comparison across tasks. A trend can show that a capability has improved without showing that models are equally capable in all settings. A comparison can show that one type of task is better handled than another without revealing how either has changed over time. Treating the two findings as interchangeable would obscure what METR’s report, as described in the excerpt, actually says.
That distinction matters when coding results are used to discuss broader model competence. The comparison gives readers a reason not to assume that performance on coding tasks carries over to tasks requiring judgment. It does not, on its own, establish how a model would perform on any particular judgment-heavy assignment. For that, a reader would need to know how the tasks in the report were defined and how performance was assessed. The excerpt provides neither, so the comparison is informative about the direction of the gap but not about its practical size.
The same caution applies in the other direction: the judgment finding does not cancel out the reported exponential gains. The two observations can coexist because they answer different questions. A model can perform better on one type of task than another while some of its capabilities continue to improve. What cannot be determined from the excerpt is whether the capabilities behind the exponential trend include coding, judgment, or both. Keeping that question open is more faithful to the report as summarized than turning either finding into a blanket statement about AI progress.
Frequently Asked Questions
What did METR update?
METR updated its time horizon plot in a new report covered by Grace Huckins for MIT Technology Review. The available excerpt identifies the plot and summarizes two findings, but it does not show what changed in the graph or explain how to read it.
Does the report say all AI capabilities are improving exponentially?
No. The excerpt says that some AI capabilities continue to improve along an exponential trajectory. It does not identify those capabilities, give a rate of improvement, or say that the same trend applies to every kind of task.
How large is the performance gap between judgment and coding?
The excerpt does not quantify the gap. It says that models do not perform as well on tasks requiring judgment as they do on coding tasks. It does not provide the scores, task definitions, or evaluation details needed to judge the size of that difference.


