Can AI’s ‘Scaling Laws’ Continue to Drive Improvement—Indefinitely? History Suggests Caution.
Table of Contents
The relentless pursuit of bigger and more powerful AI models, fueled by the promise of “scaling laws,” may be hitting the limits of practicality—and even mirroring historical engineering failures.
OpenAI CEO Sam Altman, a leading figure in the current artificial intelligence boom ignited by the launch of ChatGPT in 2022, is a firm believer in scaling laws. These rules of thumb, linking AI model size to capability, are driving massive investment in computing power, data centers, and even nuclear energy to meet the growing demands of AI development. The core idea, as Altman articulated in a recent blog post, is that an AI model’s “intelligence” is directly proportional to the resources invested in its training and operation—suggesting continuous improvement through exponential scaling.
First observed in 2020 and refined in 2022, these scaling laws for large language models (LLMs) provide a seemingly simple formula for engineers, predicting performance gains based on model size. However, a growing chorus of experts and historical precedents suggest that these laws may not be immutable, and the current investment frenzy could be built on shaky foundations.
Scaling Laws: Not Unique to AI
The concept of scaling laws isn’t exclusive to artificial intelligence. Modern aerodynamics, for example, relies heavily on them. Engineers utilize the Buckingham π theorem to accurately compare scaled-down models in wind tunnels with their full-scale counterparts, ensuring consistent performance characteristics. These principles underpin the design of aircraft, ships, and even industrial machinery.
Similarly, Moore’s Law—the observation that the number of transistors on a microchip doubles approximately every two years—fueled decades of innovation in computing. This principle allowed for the creation of increasingly small and powerful technology. However, it’s crucial to recognize that not all “scaling laws” are inherent laws of nature. Some are purely mathematical constructs, while others are empirical observations that hold true only within specific parameters.
When Scaling Laws Break Down
History is replete with examples of scaling laws failing spectacularly. The collapse of the Tacoma Narrows Bridge in 1940 serves as a stark reminder. The bridge, designed by scaling up successful smaller bridge designs, succumbed to aeroelastic flutter due to unforeseen instabilities caused by wind. Engineers incorrectly assumed that the same scaling principles would apply to a longer, slimmer structure.
The limitations of scaling also became apparent in microchip manufacturing. While Moore’s Law and Dennard scaling (increasing transistor density while maintaining power consumption) were remarkably reliable for decades, they eventually encountered physical limits. As transistors shrank to the nanoscale, current leakage and unpredictable behavior emerged, rendering further miniaturization impractical. Innovation shifted from simply shrinking transistors to developing new chip architectures.
Laws of Nature or Rules of Thumb?
The scaling curves currently guiding LLM development are demonstrably real and have proven useful thus far. They have confirmed that increased data and computing power lead to improved model performance and revealed that earlier systems were not fundamentally limited, but rather under-resourced. However, these curves are ultimately fitted to data, making them more akin to the rules of thumb used in microchip design than the mathematically derived laws of aerodynamics. This suggests their applicability may be finite.
These scaling rules fail to account for real-world constraints, such as the limited availability of high-quality training data, the challenges of adapting AI to novel tasks, and the significant safety and economic hurdles associated with building and powering massive data centers. There is no inherent guarantee that “intelligence” will scale indefinitely.
The Financial Reality of AI Scaling
While the scaling curves for AI appear smooth, the financial implications are far more complex. Deutsche Bank recently warned of a potential AI “funding gap,” estimating an $800 billion mismatch between projected AI revenues and the necessary investments in chips, data centers, and power infrastructure.
JP Morgan estimates that the AI sector may require around $650 billion in annual revenue to achieve a modest 10% return on the planned infrastructure build-out. The future trajectory of LLMs remains uncertain. Current scaling rules may continue to hold, or new bottlenecks—related to data, energy, or user adoption—could disrupt the trend.
Altman is betting on the continuation of LLM scaling laws, arguing that predictable gains justify massive investments in computing power. However, the growing concerns from financial institutions serve as a cautionary tale, reminding us that seemingly promising scaling stories can end like the Tacoma Narrows Bridge—beautiful in theory, but disastrous in practice.
This article is republished from The Conversation under a Creative Commons license. Read the original article.
Citation: Can bigger-is-better ‘scaling laws’ keep AI improving forever? History says we can’t be too sure (2025, November 29) retrieved 29 November 2025 from https://techxplore.com/news/2025-11-bigger-scaling-laws-ai-history.html
