Why the Newest AI Models Take Longer to Answer, On Purpose

Ask a standard large language model a hard math problem and it answers almost instantly, whether or not the answer is correct. Ask one of the newer reasoning models the same question and it may sit for ten, twenty, sometimes sixty seconds before responding, visibly working through intermediate steps rather than producing an answer in one pass. That pause is not a performance regression. It is the point.

We have spent most of the last several years thinking about AI capability as primarily a function of training: bigger models, more parameters, more pretraining compute, better data. Reasoning models built around test-time compute, sometimes called inference-time scaling, add a second, largely independent lever. Rather than only investing compute while the model is being trained, these systems invest additional compute at the moment they are actually answering a question, and that additional thinking has produced real, measurable gains on tasks that pure scale alone had struggled with.

A Second Scaling Axis, Not a Replacement

OpenAI’s o-series models were among the first widely deployed systems built explicitly around this idea, and the results on genuinely hard problems have been notable. Reporting on o3’s performance describes breakthrough results on the ARC-AGI abstract reasoning benchmark, alongside strong gains on math and coding tasks that had previously resisted improvement from scale alone. Google’s Gemini Deep Think reportedly reached International Mathematical Olympiad gold-medal standard in mid-2025, solving five of six problems, a result that would have been difficult to imagine from a model answering in a single pass.

This is best understood as an additional scaling axis rather than a replacement for pretraining scale. The parallel push toward smaller, more efficient models trained through distillation addresses a different part of the cost equation entirely, shrinking the model itself, while test-time compute scaling leaves the model’s size largely alone and instead changes how much work it does per query. The two approaches are complementary rather than competing, and some of the most capable systems now combine a smaller, well-distilled base model with the option to reason longer when a specific query demands it.

How a Model Actually “Thinks” Longer

Under the hood, researchers generally describe two broad approaches to spending extra compute at inference time. Sequential scaling has the model iteratively refine its own answer, generating an initial chain of reasoning, checking it, and revising before committing to a final response, a form of structured self-reflection built into the generation process itself. Parallel scaling instead generates multiple independent candidate solutions at once and uses a separate verification step, often another model, to select the strongest one, an approach commonly described as best-of-N sampling.

Both approaches share a common trade-off: more reasoning steps or more candidate paths mean more tokens generated before the user ever sees a final answer, and every one of those intermediate tokens costs compute and time, whether or not the user ever reads them directly.

What the Extra Thinking Actually Costs

The latency difference between a standard model and a reasoning model working through a hard problem is not subtle. Industry commentary on current-generation reasoning systems generally puts standard model response times at one to five seconds, against ten to sixty seconds or more for a reasoning model working through a genuinely difficult query, and pricing for these systems typically counts the intermediate reasoning tokens toward the bill, not just the final visible answer. For a simple factual question, routing to a reasoning model at full effort can mean paying an order of magnitude more for a response that a fast model would have answered correctly anyway.

That asymmetry has made query routing a genuine engineering discipline rather than an afterthought. The practical rule of thumb we see reflected across builder discussion is straightforward: reasoning models earn their cost when the price of a wrong answer exceeds the price of the extra latency and tokens, math, multi-step coding tasks, and problems with a verifiable correct answer being the clearest cases, while simple factual lookups or syntax questions rarely justify the wait.

Teaching Models to Think Only as Much as They Need To

A newer wave of research has focused specifically on avoiding wasted deliberation, since a reasoning model applying full extended reasoning to every query, including trivial ones, is both slow and unnecessarily expensive. Approaches described in recent research favor compact, lightweight reasoning for routine requests, escalating to fuller chain-of-thought deliberation only when a query’s difficulty actually warrants it, an adaptive middle path between always reasoning and never reasoning. Other recent work has reported meaningful reductions in reasoning-token usage, in some cases roughly halving token spend relative to fixed, always-on high-effort reasoning, with limited measured impact on task success.

This adaptive framing matters because it reframes the core design question. It is not simply whether to use a reasoning model, but how to build a system capable of recognizing, on a per-query basis, how much deliberation a given problem actually requires, and spending compute accordingly rather than applying a single fixed setting to every request that arrives.

What This Means for Teams Building on These Models

The broader question of how far current reasoning systems can actually be pushed on long, multi-step tasks remains genuinely open, and test-time compute scaling has not eliminated the limitations that show up once a task requires sustained reasoning across many steps rather than a single hard problem. What it has done is give builders a real, usable lever distinct from model size: the ability to trade latency and cost for accuracy on a per-query basis, deployed selectively rather than applied uniformly.

For teams building products on top of these models, the practical work is less about picking a single model and more about designing routing logic that matches problem difficulty to reasoning depth, and being honest with users about the trade-off being made when a response takes forty seconds instead of two. How much of that additional accuracy is genuine reasoning gain versus an artifact of how these systems are being benchmarked is a separate, still-unsettled question, but the operational reality for anyone shipping AI products today is more immediate: knowing when to let a model think longer has become as important as knowing which model to use in the first place.