Smaller Models, Distilled Teachers: How Training Efficiency Became the New Scaling Law

For much of the last several years, the dominant story in AI capability was scale: bigger models, trained on more data, with more compute, produced better results, and that relationship held reliably enough to organize entire research strategies around it. A quieter, increasingly important story has been developing alongside it. Model distillation, the practice of training a smaller model to reproduce the behavior of a larger one, has moved from a cost-saving trick to a genuine driver of capability, and it is reshaping how new models actually get built.

What Distillation Actually Does

The basic mechanism is straightforward to describe even though the engineering behind it is not. A large “teacher” model generates outputs, or is used to guide training in more sophisticated ways, and a smaller “student” model is trained to match that behavior rather than being trained from scratch on raw data alone. The result, done well, is a smaller model that captures a meaningful share of the teacher’s capability on the tasks it was distilled for, at a fraction of the parameter count and inference cost.

What has changed recently is how much capability that process can actually transfer. Distilled models in the roughly three to fourteen billion parameter range are now matching the performance that considerably larger models delivered only a year or two earlier, on tasks like classification, summarization, structured extraction, and code completion. That is a meaningfully different claim than “smaller models are cheaper but weaker.” It is closer to “smaller models trained well now do most of what larger models did recently, for specific, well-defined tasks.”

Why This Counts as a Genuine Scaling Law, Not Just a Cost Optimization

It would be easy to file distillation under efficiency and move on, but that undersells what is happening. Traditional scaling laws described a relationship between model size, data, compute, and resulting capability. What distillation research is increasingly showing is a second, related relationship: how much of a large model’s capability a smaller model can absorb depends heavily on training methodology, not just on the student model’s raw parameter count.

That reframes the central question in a useful way. Instead of asking only “how big does a model need to be to do this task,” researchers are increasingly asking “how well can a smaller model be taught to do this task,” and the answer to the second question has been improving faster than raw parameter-count scaling alone would predict.

Where the Frontier Labs Are Actually Using This

Distillation is no longer a niche technique reserved for cost-constrained deployments. It has become a standard part of how frontier labs build their own smaller model tiers, training compact models specifically to inherit reasoning and instruction-following behavior from larger siblings within the same model family, rather than training smaller models independently from scratch. Research comparing small models trained purely with reinforcement learning against small models distilled from a stronger teacher has generally found the distilled versions more sample-efficient and more capable for a given size, which is part of why this approach has become close to the default rather than an alternative path.

What This Means for the Long-Horizon Problem

This research trend connects directly to one of the harder open problems in applied AI right now. Even highly capable models still struggle to reliably complete long, multi-step tasks without drifting off course, and a natural question is whether smaller, well-distilled models can help with that problem by handling well-defined sub-tasks reliably within a longer agentic workflow, freeing a larger, more expensive model to handle only the steps that genuinely require broader reasoning. Early evidence suggests this kind of layered approach, distilled specialists handling routine sub-tasks while a larger model handles the harder judgment calls, is a promising direction, though it introduces its own coordination challenges that are still being worked out.

The Trade-Offs Nobody Should Skip Past

None of this makes distillation a universal solution. Distilled models still lag meaningfully behind their teachers on open-ended reasoning, tasks requiring very long context retention, and niche factual knowledge that was not well represented in the distillation process. A model distilled to excel at customer support ticket classification is not a substitute for a frontier model handling genuinely novel research questions, and treating a well-distilled small model as a drop-in replacement for a large one across every task remains a reliable way to be disappointed by the results.

This same tension between overall benchmark performance and task-specific reliability shows up in a broader measurement problem the field has been grappling with. Aggregate benchmark scores increasingly fail to capture how a model performs on the specific task an organization actually cares about, and distillation makes that gap even more consequential, since a distilled model’s performance can vary sharply depending on how closely a real-world task resembles the tasks it was actually distilled for.

What to Watch Next

The practical implication for anyone building on top of these models is that the size of a model is becoming a less reliable proxy for its capability on any specific task than it used to be. A smaller, carefully distilled model built for a narrow purpose can outperform a much larger general-purpose model on that exact task, while remaining far behind it on anything outside that scope. Evaluating models on the actual task at hand, rather than defaulting to the largest available option, is becoming a genuinely important skill, not just a cost-saving habit.

Training efficiency research is still moving quickly, and the specific techniques driving today’s distillation gains are likely to look dated within a couple of years, the same way earlier scaling approaches did. What seems durable is the underlying shift in framing: capability is no longer purely a function of size, and how a model was taught is becoming as important a variable as how large it is.