Moore's Law says the number of transistors on a chip doubles about every two years. For fifty years it set the pace of the computer industry.
The compute used to train the largest AI models has grown far faster. Epoch AI, a research group that tracks AI systems, estimates that training compute for notable models has grown 4 to 5 times every year since 2010. That is a doubling roughly every five to six months.
The largest training run each year
Compute used by the largest AI training run of each year
Floating point operations (FLOP), log scale. Each step on the axis is ten times the one below
The dashed line shows what Moore's Law growth, one doubling every two years, would have delivered from the 2010 level. Some recent values are Epoch estimates marked speculative.
Source: Epoch AI, Data on Notable AI Models (CC BY 4.0), accessed 3 October 2026
In 2010 the largest training run used about 5 × 1016 operations. Epoch's estimate for the largest run published by September 2026 is 1027. That is an increase of about 18 billion times in 16 years.
Moore's Law over the same years would have delivered an increase of 256 times.
Landmark models
Training compute of landmark AI models
FLOP, as estimated by Epoch AI
| Model | Released | Training compute (FLOP) | Epoch confidence |
|---|---|---|---|
| AlexNet | Sep 2012 | 4.7 × 1017 | Confident |
| Transformer | Jun 2017 | 7.4 × 1018 | Confident |
| BERT-Large | Oct 2018 | 2.9 × 1020 | Confident |
| GPT-3 | May 2020 | 3.1 × 1023 | Confident |
| PaLM | Apr 2022 | 2.5 × 1024 | Confident |
| Llama 3.1 405B | Jul 2024 | 3.8 × 1025 | Confident |
| Grok 4 | Jul 2025 | 5.0 × 1026 | Speculative |
Source: Epoch AI, Data on Notable AI Models
AlexNet, the image model that started the deep learning boom, used about 5 × 1017 operations. Llama 3.1, released twelve years later, used about 80 million times as much.
Where the extra compute comes from
Faster chips explain only part of the growth. Three other things did the rest.
- More chips per run. Training now runs on tens of thousands of GPUs or TPUs working together.
- Longer runs. Large models train for weeks or months.
- More money. Epoch estimates the cost of frontier training runs has grown about 2 to 3 times a year.
Each of these has limits. Power supply, chip production and budgets will decide how long the trend can last. For now it shows no sign of matching the slower pace of Moore's Law.