The AI Scaling Law and DeepSeek

By Maia Salti

Endless GPUs generated by Gemini Nano Banana

NVIDIA's Crash

On January 27th, 2025, NVIDIA lost 589 billion US dollars in market cap in one day, the largest one-day loss for any company in US history, beating its own prior record of 279 billion in September 2024. The broader tech/AI market lost about 1 trillion USD that day.

Compute

Initially a company that sold computer chips for game rendering, NVIDIA now sells GPUs to the companies that create the frontier models and provide them with the compute they need to function. Compute has been a scarce asset for many of these competing companies (Anthropic, OpenAI, Google, Meta).

The Scaling Law

Model capability has been known to scale predictably with compute.

If you increase compute, model capability will increase. This is what led to companies focusing on simply making their models much bigger: feeding them more data, providing more compute, and adding more parameters.

The Consequences of Believing the Scaling Law

Based on the assumption that the scaling law was the only truth, the U.S. controlled the export of advanced GPUs in order to deny frontier AI to other countries, like China. The market also adjusted to this truth, a belief that skyrocketed NVIDIA’s valuation, because of how much compute it provided to the US AI companies (GPT-3 trained on NVIDIA V100, Llama 2 on A100, Claude 2 on A100).

U.S. Export Control

DeepSeek, the Chinese AI company trying to compete with the US frontier models, didn’t have the same access to the GPUs: it was trained on NVIDIA’s H800s. H800s are a China-specific version of the H100 designed to comply with the U.S. export restrictions, which had reduced interconnect performance (meaning slowed communication between GPUs).

How Scarcity Forced Efficiency

Mixture of Experts

A normal model runs every parameter for every token it processes. If the model has 175 billion parameters (GPT-3), it will need to run all 175 billion of those parameters for each word, which is very computationally expensive.

DeepSeek, constrained by compute, sought to work inside of the box. They focused on the Mixture-of-Experts model (MoE) which, instead of running all parameters for every token, picks only a few relevant “expert” parameters to activate, and leaves the rest asleep. So DeepSeek-V3 has about 671 billion total parameters in memory but only about 37 billion activate per token (5%). The payoff is that you get the knowledge capacity of a huge model while only paying the compute cost of a smaller one.

The equivalent would be a patient who broke their arm seeing every specialist in the hospital (GPT-3) versus only seeing an orthopedist (DeepSeek).

Multi-head Latent Attention

When a model creates text, it has to keep running notes of everything it’s already processed and written (these notes are called the KV-cache). For long sequences this takes up a lot of memory, which matters a lot when the H800 GPU, which is slower at communicating, needs to move all of this data around.

MLA compresses the running notes so that data transferring becomes much easier and more efficient. DeepSeek introduced this method to address the memory bandwidth issue.

DeepSeek Defied

So DeepSeek, although it still spent compute on pre-training, spent more compute generating a long internal chain of reasoning. This allowed the model to think through complex queries rather than using a “larger pre-training brain,” and they did that through reinforcement learning (RL). It’s the equivalent of making a child memorize a textbook (pre-training-heavy compute) versus giving a child practice quizzes and telling them where they were right and wrong (reinforcement learning for inference).

Conclusion

So did DeepSeek break the scaling law? Not exactly. They showed that capability could come from variable inference-time compute and not just fixed pre-training compute, following the path o1 opened, but they did it cheaply via RL.

NVIDIA crashed in January of 2025 because the market thought that compute was no longer the moat once DeepSeek was released, and therefore the demand for GPUs decreased. But that was wrong; compute was just distributed differently. If anything, it could increase demand for compute because a pre-training cost is paid once, whereas an inference cost is paid every time anyone uses the model. So even though DeepSeek made their model run cheaper, it has potential to increase compute demand.

So the question that I have to ask is this: (export controls aside) would it be better for NVIDIA to sell chips to China, accepting the risk that abundant compute might let them reach the frontier first, with all the dangers that implies, or to restrict access and risk one of two things: that scarcity drives China to build its own competitive GPUs, or forces the kind of efficiency innovation that reaches the frontier anyway?

Sources

• Bloomberg — Nvidia's $589B DeepSeek plunge
• CNBC — Nvidia sheds ~$600B, biggest one-day drop
• Business Standard / Reuters — the $593B figure
• Bloomberg — Nasdaq slump as DeepSeek sparks concern
• Fortune — ~$1T tech wipeout
• Nature — DeepSeek-R1 paper (open-access mirror)
• The Register — What DeepSeek's training really cost
• DeepSeek-V3 Technical Report (arXiv)
• DeepSeek-R1 paper (arXiv)
• SemiAnalysis — DeepSeek Debates (true cost & fleet)
• Tom's Hardware — 50,000 GPUs / $1.6B buildout
• OpenAI API pricing
• KNIME — OpenAI o1 vs DeepSeek R1
• VentureBeat — How R1 challenges o1 on cost
• Epoch AI — Cost to train frontier models
• Stanford AI Index (via Statista) — training-cost estimates
• Sam Altman — GPT-4 ">$100M" estimate (Hacker News)
• Epoch AI — Notable AI Models database
• Our World in Data — Growth of training compute
• Model papers — GPT-3 · Llama 2 · PaLM · PaLM 2 · GPT-4 · Gemini 1.0
• Meta — Llama 3 blog · Llama models (GitHub)
• Artificial Analysis — model benchmarks · DeepSeek R1 update
• Pricing — Anthropic · Google Gemini · DeepSeek

Data research run by claude code