The Measurement Problem: Why We Can No Longer Tell How Much Better AI Is Getting

For most of the past decade, the question of whether AI systems were improving had a convenient answer: look at the scores. Benchmarks gave the field a shared scoreboard, and the numbers moved in one direction. That arrangement is now breaking down, and the way it is breaking down tells us something more interesting than the scores ever did.

The immediate symptom is saturation. Benchmarks that once separated strong models from weak ones now cluster everything at the top. MMLU, HumanEval, GSM8K and similar suites have been effectively exhausted, and newer replacements are being consumed faster than they can be built. A systematic study of benchmark saturation published in February 2026 traced this pattern across the major evaluation suites, and the trajectory it describes is not one that better test design alone reverses.

Saturation Is the Least Interesting Part

It would be tidy if the problem were simply that models had become too capable for the tests. That framing is flattering and mostly wrong.

Two other things are happening at once. The first is contamination. Benchmarks live on the public internet, which is also where training data comes from, and the boundary between the two has never been well policed. A score can rise because a model reasons better or because it has seen the answers, and from the outside these look identical.

The second is optimization pressure. When a number determines commercial positioning, effort flows toward the number. This is not necessarily fraud; it is the ordinary consequence of measuring something and attaching stakes to it. Goodhart’s observation about measures becoming targets applies here with unusual force, because the measure is public, the target is valuable, and the gap between them is difficult for an outsider to audit.

Put together, these mean that a rising benchmark score has become a weaker signal than it was three years ago, even when nothing dishonest has occurred.

The Gap That Actually Matters

The more consequential problem is the distance between benchmark performance and behavior in deployment. Reports through 2026 have repeatedly described enterprise systems performing well in evaluation and disappointingly in production, and the explanations offered tend to be the same each time: real inputs are messier, real tasks run longer, and real environments contain failure modes that a curated test set does not.

We would treat the specific figures circulating on this point with some caution. Much of the widely quoted data comes from vendor reports and consultancy surveys with undisclosed methodology, and the incentives around those numbers are not neutral. The direction is consistent enough to take seriously; the precision is not.

What is clearer is the structural reason for the gap. Benchmarks are built from tasks with known answers, short horizons, and clean inputs. Deployment involves tasks where the answer is contested, the horizon is long, and the input arrives in whatever form it happens to arrive. A system optimized against the first is not thereby optimized against the second, and the more effort goes into the first, the more the two can diverge.

What We Are Really Measuring

There is a deeper issue underneath the methodological ones. Benchmarks measure task completion. What organizations care about is reliability, which is a different property and a harder one.

A model that answers correctly ninety-five percent of the time on a benchmark is an excellent model by benchmark standards. A system that fails one interaction in twenty may be unusable in a context where failures are expensive and detection is difficult. The benchmark reports an average; deployment cares about the distribution, particularly its tail.

This distinction is why capability and usefulness have come apart in the way they have. It is possible for a system to be genuinely more capable than its predecessor and no more useful, because the improvement landed in the average rather than in the tail. We touched on a related version of this in our discussion of why interface quality is often mistaken for intelligence: the observable improvements and the underlying ones are not always the same thing.

Where Evaluation Appears To Be Heading

Several directions are visible in current work, none of them complete.

Held-out and dynamic evaluation

Keeping test sets private, rotating them, or generating them procedurally addresses contamination. It also makes independent verification harder, which is a real cost. A benchmark nobody can inspect requires trusting whoever holds it.

Reliability over accuracy

Recent work has proposed measuring variance across repeated attempts rather than single-run success, on the reasoning that a system succeeding inconsistently is a different object from one succeeding reliably. This is closer to what deployment cares about and considerably more expensive to run.

Task-length horizons

Instead of asking whether a system can do a task, some evaluations ask how long a task it can complete before failing. This reframing has proven informative for agentic systems, where duration rather than difficulty is often the binding constraint.

Domain-specific evaluation

The most practical response we see organizations adopting is to stop relying on public benchmarks entirely and build internal evaluations against their own tasks and data. This is unglamorous, does not produce a comparable number, and is generally more informative than anything on a leaderboard.

What This Means in Practice

For anyone selecting or deploying these systems, a few things follow.

Treat benchmark scores as a filter rather than a decision. They are reasonable for excluding options and poor for choosing among the remaining ones. The differences at the top of a saturated leaderboard are frequently within noise.

Build an evaluation from your own work. A few dozen representative tasks from actual operations, run against candidate systems, will tell you more than any public score. The effort is smaller than it sounds and the results are usually surprising.

Measure the failures, not the successes. How often does the system fail, how badly, and how easily is the failure noticed? A system that fails visibly is safer than one that fails plausibly, and no benchmark reports this distinction.

Account for cost alongside capability. Performance differences that are marginal can come with cost differences that are not, a dynamic we examined in our analysis of how AI’s cost center moved into production.

The Honest Position

The field is in a period where its measurement tools have outlived their usefulness and their replacements are not ready. This is uncomfortable but not unusual; most fields go through it when the thing being measured changes faster than the instruments.

The mistake would be to keep treating the old numbers as though they still carry the information they used to. They do not, and the confidence they invite is the main risk. Progress in AI is almost certainly real. Our ability to say how much, in what direction, and whether it will show up in any particular deployment is weaker now than it was, and pretending otherwise helps nobody.

The organizations we see handling this well have mostly stopped asking which model is best and started asking whether a specific system does a specific job acceptably. That is a smaller question, and at the moment it is the one that can actually be answered — a shift in posture that also applies to where human judgment still sits in an automating workflow.

If you are working through evaluation questions in your own organization, I am always glad to compare notes — you can find me on LinkedIn.