Stanford MS&E435: Economics of the AI Supercycle — Applied AI

ELI5/TLDR

Tuhin, CEO of an inference company called Base 10, sits in a Stanford class and explains the plumbing underneath every AI app you use. His company doesn’t build models — it runs them, very fast, on borrowed GPUs stitched together from 20 different cloud providers. His big bet: the smart AI apps will stop renting brains from OpenAI and Anthropic and start owning their own, by taking cheaper open-source models and tuning them. And the deeper bet underneath that one: there will never be enough compute, ever, so whoever can reliably get their hands on GPUs wins.

The Full Story

What an inference company actually does

When you talk into a voice-typing app and text appears, something has to take your audio, run it through a stack of models, and spit out words — fast. That running of the model is called inference. Think of it like the difference between writing a recipe (training the model) and actually cooking the dish every time someone orders (inference). Base 10 does the cooking, at scale, for other companies.

Its customers are AI apps you may have used: Whisper Flow (voice-to-text), Abridge (a medical “ambient scribe” that listens in the exam room and writes the clinical note), Cursor (the coding tool). Abridge alone runs about 20 different models — speech-to-text, note generation, integration with hospital record systems — and every one of them runs on Base 10.

The obvious question, which the interviewer asks: why wouldn’t these companies just go straight to Amazon or Google for their compute? Tuhin’s answer is that the raw cloud gives you a GPU and a shrug. You’re on your own to make inference fast, reliable, and able to fail over across data centers. Base 10 sells the software layer that does all of that.

What we actually find is a lot of them do go to those folks first, and then realize the pain of standing up that whole inference stack on top of compute, and realize they’ll just be better served coming to base 10.

The big bet: own your intelligence

Here’s the thesis worth slowing down on. Right now, Tuhin says, roughly 90-95% of money spent on inference goes to frontier models — the best, newest, most expensive ones from OpenAI, Anthropic, Google. Only about 5% goes to open-source models that companies tune themselves.

He thinks that ratio is going to shift, and his whole company is built on it. Open-source models, he claims, run about 90 days behind the frontier and cost 70-90% less. For a company that has found its product but now needs to make money, that gap is the difference between bleeding cash and a real business.

The leading coding companies that are not the frontier model companies themselves are still rumored to be negative gross margin.

In plain terms: some famous AI apps lose money on every query because they’re paying frontier prices for the intelligence. Switching to a cheaper, self-tuned model is how they survive.

There’s also a darker reason, which Tuhin frames vividly. If you keep building on someone else’s frontier model, you hand them all your user data and workflow signals — and eventually they can just copy the thing that made you special.

He likened the Frontier Labs to the East India Company. They show off in India, make all these partnerships, but really what they’re doing is you’re giving them all the tricks on how to rule that.

Base 10, he jokes, is the “West India Company” — arming the rebellion. The serious point underneath the joke: to stay defensible, an app company needs to own its intelligence, not rent it.

Post-training, in concrete terms

When asked to demystify “post-training,” Tuhin gives a clean example. Imagine you’re building speech-to-text for hospitals. First you define your utility function — the single thing you want to minimize, say transcription errors. Then you pick an open-source base model (he names a Chinese one, “Kimi K2.5”), hand over a dataset, and Base 10 provides the scaffolding to turn that generic model into a specialized one tuned for your exact job. Then it plugs straight into inference. You arrive with data and knowledge of your workflow; you leave with a custom model running in production.

Why the best open-source models come from China

A striking exchange. The best open-source AI models today, Tuhin says, come from China — Moonshot, Alibaba, MiniMax — not America. Why can’t America produce them? His read: the best researchers all work at two companies (the implication being OpenAI and Anthropic, “despite their names maybe suggesting otherwise”), and those companies have no incentive to give away their work. Chinese labs went open-source partly to stay relevant by taking the opposite position to the closed players.

He frames it as a national-security concern: if the cost of intelligence is 70-90% cheaper in the East than the West, that’s a bad outcome for America. He notes this lands the same week Anthropic published a post arguing the US must stay ahead of China — and pointedly adds that those labs have a massive profit motive and “shouldn’t be the arbiters of morality for the rest of us.”

The compute crunch is worse than you think

The most quotable stretch. However bad you’ve heard the GPU shortage is, Tuhin says, “it is 10 times worse.” Order a thousand GPUs today and you’re being quoted Q2 of next year — 12 to 15 months out.

His own war story: Base 10 rents a cluster of Blackwell B200 chips at $2.63 per GPU-hour. The provider came back in May, months before renewal, and said next year’s price is $5.10 — double. (He’s refusing to pay it.)

To make the scale vivid: Base 10 currently processes around 30 trillion tokens a day, which he claims is larger than OpenAI’s API business and larger than Gemini’s. Project that out two years and the company will need roughly 150,000 B200-equivalent GPUs — about $7 billion of compute. You cannot reliably rent that, which is why Base 10, after years of renting on principle, will start buying and building its own data centers. Owning is also about 30% cheaper, since you stop paying a cloud’s gross margin.

He doesn’t think the scarcity ever normalizes. Apps are getting more agentic (doing multi-step work, burning far more tokens) and models are getting bigger. Both push demand up. His analogy: an airport at 5am has no line, but by 8am it’s out the door — except inference never gets the late-night reset, because when demand sleeps in America it wakes up in Europe and China. It just compounds.

Why Nvidia still wins, for now

Despite preaching diversity, Base 10 runs most of its fleet on Nvidia. The reason isn’t just the chips — it’s CUDA, Nvidia’s software ecosystem, and a supply chain nobody else has. “There’s nothing like CUDA. CUDA is insane.” He expects the future to be heterogeneous — different chips for different parts of inference (the memory-heavy “decode” step split from the compute-heavy “prefill” step) — but for a company trying to move fast today, Nvidia’s tooling is unbeatable.

What he’d build instead

Asked his next-best idea if not Base 10: energy and power, like Crusoe. And one specific pitch he hopes a student steals — modular data centers. Just as the shipping container standardized global trade by normalizing the unit, a standard, repeatable unit of compute would let the whole industry industrialize data-center buildout. “You’re creating an API for compute.”

Key Takeaways

Inference = running a trained model to produce outputs; it’s the recurring cost of delivering AI, distinct from the one-time cost of training.
Today ~90-95% of inference spend goes to frontier models, ~5% to open-source. Base 10’s entire bet is that this flips toward self-tuned open-source models.
Open-source models run ~90 days behind the frontier and cost ~70-90% less to run.
Some leading non-frontier AI apps (e.g. coding tools) are rumored to run negative gross margins — losing money per query — making the switch to cheaper models existential.
The bigger an app company gets, the more existential post-training becomes; tiny token-trading margins don’t scale.
Strategic risk of building on frontier labs: you feed them your data and workflow signals, and they can eventually replicate your edge (“East India Company”).
The best open-source models today come from China (Moonshot, Alibaba, MiniMax), not the US — Tuhin frames the US gap as a national-security issue.
GPU shortage is severe: ~12-15 month lead times; one provider doubled Base 10’s price from $2.63 to $5.10 per GPU-hour at renewal.
Base 10 processes ~30 trillion tokens/day, which Tuhin claims exceeds OpenAI’s and Gemini’s API token volumes.
Two-year compute need: ~150,000 B200-equivalents ≈ $7 billion — too large to rent, so they’re moving toward owning data centers (~30% cheaper than renting).
Base 10 stitches compute from 20 clouds across 87 clusters to make GPUs “fungible” and hedge scarcity.
Nvidia’s durable moat is CUDA + supply chain + TSMC relationship, not just the silicon.
Future inference is likely heterogeneous: prefill and decode split across specialized chips rather than one GPU doing everything.
Tuhin’s pitch for an aspiring founder: modular, standardized data centers — “an API for compute.”

Claude’s Take

This is a sharp, candid founder interview, and Tuhin is unusually good at explaining infrastructure without hiding behind jargon. The “own your intelligence” thesis and the East India Company framing are genuinely useful mental models, and the compute-scarcity numbers are concrete enough to be worth remembering.

The BS filter, though, has to stay on for a few things. First, almost every claim here is in Base 10’s commercial interest — that open-source will catch up, that the app layer survives independently, that inference is sticky like databases. He’s honest enough to label these as his three core risks, which is to his credit, but a class interview is also a sales pitch. Second, the headline stats are unverified founder claims: “bigger than OpenAI’s API,” “30 trillion tokens a day,” “$7 billion of compute” — plausible, directionally interesting, but self-reported with no source. Treat them as the company’s framing, not facts. Third, the “open-source is 90 days behind and 70-90% cheaper” line is the load-bearing assumption of the whole business, and it’s exactly the kind of number that’s true today and could be false in a year if the frontier labs pull away or the Chinese labs stop shipping.

The most valuable parts are the texture you can’t get from a press release: the GPU price literally doubling at renewal, the guy in the corner of the office who “just calls people all day asking for compute” like a drug market, the airport analogy for why scarcity never resets. Score 7 — clear, quotable, intellectually honest about its own risks, but it’s one motivated narrator describing a market he profits from, and the big numbers come with no receipts.