For more than a decade, AI innovation has been reduced to a deceptively simple formula: more parameters, more data, more compute. This belief—that progress is primarily driven by scaling—has reshaped research culture, funding priorities, and even who gets to participate in AI research.
Sara Hooker, former Google Brain researcher and former Head of AI Research at Cohere, argues that this era of “compute worship” is nearing its end. In her article On the Slow Death of Scaling, she challenges the assumption that ever-larger models trained with ever-greater compute will continue to deliver meaningful breakthroughs.
What follows is a structured synthesis of the article’s core arguments and implications.
📉 The Rise of Small Models Can No Longer Be Ignored #
Questioning the future of scaling remains controversial. For years, increasing compute reliably produced larger models and measurable gains, fitting neatly into industry planning cycles. Proposing a bigger model often appeared safer than proposing a new algorithm.
However, recent evidence reveals a growing disconnect between model size and real-world performance. Smaller, more efficient models are increasingly outperforming much larger ones. The number of such cases has risen sharply, signaling that performance gains are no longer proportional to compute investment.
In an era of diminishing returns, what matters most is no longer absolute scale—but the performance return per unit of compute. Optimization quality, architectural choices, and data efficiency now dominate the risk–reward equation.
⚙️ What Determines the Return on Compute? #
1. Diminishing Returns from Model Scale #
Model sizes have exploded—from tens of millions of parameters to hundreds of billions. Yet the link between parameter count and generalization remains poorly understood.
A paradox persists:
- After training, large portions of model weights can be removed with little performance loss.
- But without those weights during training, the same performance cannot be achieved.
Research shows that a small fraction of weights can predict the majority of a network’s parameters, revealing massive redundancy. This suggests deep learning is fundamentally inefficient—learning long-tail, low-frequency features at enormous cost. Most training compute is spent memorizing rare patterns, an approach likened to “building a ladder to the moon.”
2. Data Quality Reduces the Need for Compute #
High-quality data consistently reduces dependence on brute-force scaling. Techniques such as deduplication, pruning, and prioritization can compensate for smaller model sizes.
This undermines the assumption that parameter count defines performance ceilings. Strategic investment in data quality can outperform raw compute expansion.
3. Algorithmic Innovation Substitutes for Scale #
Many recent gains come not from larger models, but from better techniques, including:
- Instruction fine-tuning
- Knowledge distillation
- Chain-of-Thought reasoning
- Longer context windows
- Retrieval-Augmented Generation (RAG)
- Preference and feedback-based alignment
These methods consistently improve performance at fixed compute budgets, proving that progress is increasingly about how compute is used, not how much is used.
4. Architecture Sets the Ceiling #
Architecture fundamentally shapes the relationship between compute and performance. New architectures can invalidate existing scaling assumptions entirely.
What scaling laws describe today may become irrelevant tomorrow if architectural paradigms change.
📐 The Fragility of Scaling Laws #
Scaling Laws gained influence by promising predictability: more compute yields predictable gains. This narrative justified massive capital investment and policy decisions.
In practice, these laws reliably predict only pre-training loss, not downstream task performance. Once models are evaluated on real-world tasks, results become erratic. “Emergent abilities” often serve as post-hoc explanations for failed predictions, implicitly admitting that scaling laws cannot foresee outcomes.
Compounding the issue:
- Each data point represents a full model run
- Sample sizes are tiny
- Small errors compound during extrapolation
As a result, statistical support for scaling claims is fragile and highly domain-dependent. While some tasks (like code generation) follow relatively stable trends, many others do not.
Scaling Laws may help with short-term planning under fixed assumptions, but over longer horizons, they repeatedly fail—revealing that compute stacking is not a reliable path to sustained progress.
🚀 Rethinking the Path Forward #
Compute has long been treated as a silver bullet. That assumption is breaking down.
While near-term progress will still squeeze gains from existing architectures, the compute–performance relationship is becoming tighter, noisier, and more unpredictable. Future leaders in AI will not rely on compute alone—they will reshape the optimization landscape itself.
🧰 New Optimization Spaces Are Emerging #
A growing share of compute is now spent at inference time, not during training. Techniques such as:
- Search-based reasoning
- Tool usage
- Multi-agent collaboration
- Adaptive computation
can dramatically improve performance without retraining models. Crucially, these methods bypass gradient-based learning entirely, marking a departure from three decades of training-centric AI.
At the same time, data is no longer static. Cheap synthetic data allows targeted amplification of rare but critical scenarios, breaking long-held IID assumptions and aligning models more closely with real-world demands.
🧠 From Strong Models to Intelligent Systems #
The focus of AI is shifting from “stronger models” to systems that interact effectively with the world. Interfaces, interaction loops, and system-level coordination are becoming first-class research concerns.
Problems once considered UX or HCI are now central to defining intelligence ceilings.
🧱 Why Scaling Within Transformers Is Running Out of Road #
As long as Transformers remain the dominant architecture, further scaling delivers shrinking returns. Global parameter updates struggle with continuous learning and catastrophic forgetting, making specialization and long-term adaptation difficult.
A genuine leap forward likely requires entirely new architectures, especially as AI systems move toward persistent, world-interacting operation.
Importantly, declining returns on training compute do not imply reduced environmental impact. Even smaller models, when deployed at massive scale, can drive rising energy consumption—often dominated by inference rather than training.
The central message is clear: compute alone is no longer the engine of progress. The next era of AI will be defined by architecture, data dynamics, inference-time intelligence, and system-level design—not by parameter counts alone.