The Idiot Index of Tokens

By Maia Salti

Generated by Nano Banana

The Idiot Index

In 2001, Elon Musk flew to Russia to buy refurbished ICBMs. He wanted to send a small greenhouse to Mars, cheaply, as a publicity stunt to reignite public interest in space travel. The Russians quoted him a price that he thought was absurd, and he flew home without a rocket.

A young Elon Musk at SpaceX

On the flight back, he did the math and thought about what a rocket is actually made of: aluminium, titanium, copper, and carbon fiber. He realised that the raw materials that go into a rocket cost around 2% of what SpaceX customers were being charged for a finished one. If materials were 2% of the price, the other 98% was everyone else's margin, decades of an industry that had never been forced to get cheaper.

That calculation is the reason SpaceX exists. And, according to Walter Isaacson's 2023 biography, Musk started using a phrase to run the company: the Idiot Index.

"If the ratio of finished-product cost to raw materials — the idiot index — is high, you're an idiot."

The higher the ratio, the more of the price is just markup: someone else's labor, someone else's margin, someone else's inefficiency, priced in because you had no better option.

Musk used it as a constant, almost obnoxious test inside SpaceX.

He apparently once asked an analyst about the cost of a component on the Raptor engine and was quoted something like $13,000. He responded: "It's just steel. It's about two hundred bucks."

He reportedly told his team that if you walked into a meeting without knowing which parts of your rocket had a high idiot index, you shouldn't expect to still have a job. Valves quoted at $250,000 got built in-house for a fraction of the price. An actuator quoted at $120,000 got built by an engineer for $5,000, because it turned out to be "no more complex than a garage door opener." A NASA-style latch quoted at $1,500 got replaced with $30 of bathroom-stall hardware.

None of this meant labor or assembly should be free. It meant: if the number is high enough (supposedly above 10X), you're not paying for the difficulty of the thing anymore. You're paying because nobody has bothered to make it cheaper yet, and you didn't have anywhere else to go.

Applying It to Tokens

I started thinking about that idea in the context of something completely different: what I pay per token to talk to an AI model.

The idiot index doesn't translate perfectly of course. A rocket part has raw materials you can weigh. A token doesn't have a "material" in that sense; what it has is compute.

Every token you get back from an API call was produced by a GPU spending some fraction of a second doing math, burning some fraction of a cent of electricity, on hardware someone else paid for. That's the closest thing AI has to "raw materials."

So here's the version of the idiot index I will apply:

Idiot Index of a token = price you're charged per token ÷ the marginal compute cost to generate that token.

I have to be honest before going further: that denominator is genuinely hard to pin down. The two biggest inputs, how fast you assume GPUs depreciate and how efficiently a provider is batching requests, can each swing the estimate by somewhere between 2x and 100x on their own.

The most-cited public estimate for the GPT-4 era, from the analyst firm SemiAnalysis, put compute cost around $4.90 per million tokens on a cluster of 128 A100 GPUs in mid-2023. On today's hardware, published benchmarks put the cost of running a similarly-sized model anywhere from about 2 cents to a few dollars per million tokens, depending entirely on how efficiently that hardware is being used.

So every ratio I quote below should be read the way Musk probably meant his to be read in the moment: not a precise audit, but a gut-check on whether the price makes any sense.

Token Pricing Over Time

Here's the thing that surprised me most once I actually pulled the pricing history: it looks more like a heart monitor than it does a chart of a market maturing.

If token prices reflected the arc that every other computing resource has followed (expensive at launch, then a smooth decline as the technology commoditizes), you would expect a descending line for each company.

Instead, every company's most expensive currently-available model shoots back up at almost every major release, even as its older, cheaper tiers kept getting cheaper alongside it.

Anthropic cut Opus from $75 to $25 in November 2025, its biggest price cut ever, then launched Claude Fable 5 seven months later at double that. DeepSeek compressed the same zigzag into under two years: R1 launched at nearly 8x its predecessor, got undercut by V3.2-Exp, then V4 Pro spiked again before a "temporary" 75% discount quietly became permanent.

I think this shows that the price of a token won't ever shrink toward a single value, for as long as the models are still improving. It resets to a high price every time a company ships a capability nobody else has yet, then decays back down once competitors (or a cheaper version of the same model) catch up.

It seems to be what Musk was describing. When SpaceX had no alternative to the Russian ICBM quote, the idiot index was whatever the market would bear. The moment SpaceX (or, here, a competing lab, or an open-weight alternative) could build the part itself, the price collapsed. A new AI capability behaves like a part nobody else can manufacture yet, for exactly as long as that's true.

The Idiot Index, Today

So what does the Idiot Index of the current generation of flagship models look like?

Rather than turn "price ÷ compute cost" into a single number per model, I calculated it at both ends of that 2-cent-to-$4.20 range: assuming compute costs as much as $4.20 for the lower bound of the Idiot Index, and assuming it costs as little as 2 cents for the upper bound. I don't think that either end is more "correct" than the other; the honest number for any of these models sits somewhere in between, and I don't have the data to say exactly where.

There's one more thing worth noting: falling token prices don't necessarily mean a falling idiot index. SemiAnalysis has reported that Anthropic's inference-specific gross margin rose from roughly 38% in 2025 to around the high-60s to low-70% range by mid-2026, even as list prices for Claude models fell. The efficiency gains from newer hardware and better serving software appear to be outrunning the price cuts. Because the technology is still developing, the cost of the "raw materials" is transient. The ratio, if anything, might be getting worse, even while your monthly bill goes down.

I went and pulled the cleaner version of this same question: the actual $/hour it costs to rent the GPU itself. A100s ran about $1.50/hour in 2023. H100s were around $2 by the end of that year. Then the AI boom hit, and that same H100 spiked to $8-9/hour by mid-2024, before crashing back down to about $3 now. Even the raw material didn't get steadily cheaper. It zigzagged too.

Would Musk Build His Own Rocket Here?

In Musk's framing, a high idiot index should be used as a signal that the product is worth building yourself. And that's arguably exactly what's been happening around AI pricing for the last two years. Every time a frontier lab's idiot index gets uncomfortably high, someone builds the in-house version: an open-weight model, a cheaper competitor, a company standing up its own inference stack instead of paying list price. DeepSeek's entire origin story, in this framing, is a rocket company startled by a Russian ICBM quote.

I don't think token prices are going to converge to 1x anytime soon, and honestly, I'm not sure they should; the R&D that produces a genuinely new capability has to get paid for somehow, and a training run that costs hundreds of millions of dollars doesn't amortize itself.

But I do think the idiot index is a genuinely useful lens for reading the next twelve months of AI pricing news, in a way that "the price went down" alone isn't. The real question, every time a new flagship model launches at a price that makes you sweat, isn't whether it'll get cheaper. It's how long it'll take before someone decides it's just steel, and about two hundred bucks.

Sources

• Isaacson, Walter — Elon Musk (2023), the "idiot index" quote and Raptor engine / valve / actuator / latch anecdotes
• Wait But Why — How (and Why) SpaceX Will Colonize Mars (2015, the Russia-trip origin story and the ~2% raw-material figure)
• OpenAI — API pricing · model deprecations
• Anthropic — Claude pricing
• Google — Gemini API pricing
• DeepSeek — API pricing
• SemiAnalysis — GPT-4 architecture, infrastructure, and cost (Jul 2023) · InferenceX benchmark · Anthropic growth and Bedrock mix (May 2026) · AI value capture (2026)
• NVIDIA — Lowest token cost, AI factories (Apr 2026)
• Epoch AI — Inference economics of language models (Jun 2025) · LLM inference price trends (Mar 2025)
• a16z — LLMflation: LLM inference cost (Nov 2024)
• DeepSeek's disclosed inference economics — Computerworld coverage (Mar 2025)

Data research run by Claude Code