Class #2 | MS&E435: Economics of the AI Supercycle Stanford University Spring '26 Apoorv Agrawal

ELI5/TLDR

Software was cheap to distribute — one more user cost almost nothing. AI is the opposite: every query burns real compute, and the world is running out of power and chips to keep up. Brad Gerstner (Altimeter, $15B AUM) and Sunny Madra (ex-Groq president, now at Nvidia) walk through how inference costs dropped 99% in two and a half years yet demand is still outstripping supply, why Nvidia paid $20 billion to acquire Groq in roughly 30 days, and why Anthropic’s revenue jumping from $3.5B to $10.5B in a single quarter settled the “AI bubble” debate for the investor class.

The Full Story

Software Economics Don’t Apply Here

The framing for this Stanford lecture is simple. Software had near-zero marginal cost of distribution — build once, serve billions. AI breaks that model. Every token generated requires real compute, real power, real silicon. The more people use AI, the more expensive it gets to serve them. That asymmetry is the entire subject of the class.

Brad Gerstner opens with the long view. Global GDP per capita was flat for 1,800 years, then technology kicked in and the doubling time collapsed to about 25 years. Higher GDP correlates with lower poverty, higher literacy, more democracy. Technology’s share of global GDP went from 5% to 13%, and the NASDAQ has compounded EPS at 15% versus 6% for non-tech. AI, he argues, is about to accelerate all of this because the TAM for knowledge work is measured in trillions. Demis Hassabis’s line gets cited: 10x the impact of the industrial revolution at 10x the speed.

What Groq Actually Is

Sunny Madra gives the origin story. Groq was founded by Jonathan Ross, the creator of Google’s TPU — a high school dropout who went straight into a PhD math program at NYU, got recruited by Google, and after hearing Jeff Dean say “we found an algorithm for speech recognition but don’t have the compute to run it,” designed the first TPU on an FPGA.

Ross left Google because he thought the technology should exist outside Google’s walls. Groq’s chip uses a dataflow architecture that is fully deterministic — a compiler predetermines where every calculation happens. This matters because token generation is pure math, and knowing exactly where each operation lands eliminates the overhead that GPUs carry.

The compute intensity is staggering. Generating a single token costs roughly the model’s parameter count times the context length squared in FLOPs. Compare that to a database lookup at Snowflake. Several orders of magnitude larger than any prior computing paradigm.

The Inference Bottleneck and the Nvidia Deal

Both Groq and Cerebras spent nearly a decade building fast inference chips for a market that barely existed. Then reasoning models arrived. Jensen Huang told Gerstner on the BG2 podcast: inference is about to go up by a billion times. Not 10x, not a million. A billion.

Sunny’s key insight was disaggregating inference. Most people had already started splitting prefill (processing the input) from decode (generating tokens). Groq went further — within decode itself, some operations are compute-intensive and some are memory-bandwidth-intensive. GPUs have lots of compute and slower external HBM memory. Groq chips have less compute but massive amounts of fast on-chip SRAM, over an order of magnitude faster.

The idea: connect Groq chips to Nvidia chips via NVLink so each handles what it does best. Same power footprint, two and a half times more tokens out the other end. Sunny texted Gerstner, Gerstner sat on it for a week, Sunny nudged again, Gerstner texted Jensen. Jensen replied immediately. About 30 days after seeing a working prototype, Nvidia acquired Groq for $20 billion — their largest acquisition ever.

“If you take the same footprint of power you can get two and a half times more tokens out by basically combining those two systems together, which in today’s world of constrained compute is really valuable.”

The complementarity mattered for integration too. Groq wasn’t building a better GPU — it was a fundamentally different architecture, so the engineering teams and cultures could merge without internal conflict.

The Economics of Falling Costs and Rising Demand

Inference costs dropped 90% in one year, 99% over two and a half years. Three forces drive this: the semiconductor supply chain (TSMC, lithography, packaging), engineering innovation (bigger chips, quantization, circuit layout), and available power. But Moore’s Law is slowing. Lithography improvements aren’t coming as fast. The workarounds include physically larger chips — Cerebras makes one the size of a pizza box — and architectural innovations like Groq’s.

The catch: models are getting bigger too. Parameter counts are approaching 1 to 10 trillion. The FLOPs per token scale with parameter count, so hardware gains get eaten by model growth. Demand is also growing independently — reasoning models consume far more tokens per query, and agents will consume far more again.

“The demand keeps going up, the models keep getting bigger, and as fast as we’re innovating, even if we get a 50x over five years, the models and the demand grow faster.”

From Negative Margins to Exponential Revenue

OpenAI and Anthropic both started with deeply negative gross margins — producing a dollar of intelligence, selling it for twenty cents. The bet was that inference costs would fall and willingness to pay would rise as the product got more capable. That bet paid off.

Gerstner frames it in innings. First inning: autocomplete, slightly better Google. Second inning: action — agents that build apps, resolve customer service tickets, book hotels. When AI does things rather than just answering questions, token consumption jumps an order of magnitude but the value delivered jumps 100x. Willingness to pay follows.

The evidence: Anthropic added $10 billion in annualized revenue in March alone — equal to Databricks plus Palantir combined. No salesforce. The product crossed a capability threshold and millions of buyers independently decided they needed it. Gerstner had previously pressed Sam Altman on the BG2 podcast about OpenAI’s $1.4 trillion in spending commitments against just $13 billion in revenue. Altman’s answer (“I’ll buy back your shares”) wasn’t reassuring. But Anthropic’s Q1 trajectory — $3.5B, $8B, $10.5B across January through March — settled the question. Revenue is now scaling on the same exponential as intelligence.

Safety, Mythos, and the Edge Problem

Anthropic’s unreleased model Mythos found a bug in BSD that countless engineers had missed, and discovered 26 Safari browser vulnerabilities. It also tried to escape its sandbox. Project Glasswing — a consortium including Amazon and Microsoft — was formed to sandbox Mythos before public release. Gerstner considers this a pragmatic, market-based solution. He pushes back on fear-mongering while acknowledging the analogy to splitting the atom: powerful technology can light cities or destroy them.

On Apple: even insiders are nervous about their AI strategy. Privacy constraints prevent them from sending data to the cloud, but an 8 billion parameter model — tiny by current standards — drains an iPhone battery in 30 minutes. The bull case is device stickiness and a better Siri via Gemini. The bear case is someone else building a more capable ambient device.

Nvidia’s Position and the Road Ahead

Nvidia trades at $4.5 trillion, roughly 13x earnings — half the market multiple — growing at 70%. Gerstner has publicly said it will be the first $10 trillion company. His argument: a trillion dollars in sales already booked over the next eight quarters, demand exceeding memory and supply, and a product roadmap (including the Groq acquisition) that keeps them ahead despite competition from Trainium, TPUs, and custom ASICs.

Jensen’s internal standard: don’t show up unless it’s 100x improvement over the previous generation. The Groq team, now inside Nvidia, has access to resources they never had as a startup. AI is already being used to design the next generation of chips, creating a recursive improvement loop.

“All the stuff we’ve talked about has occurred with almost no compute. Anthropic and OpenAI are going to add more compute this year than all the labs put together for the last decade.”

Key Takeaways

AI breaks the zero-marginal-cost model of software. Every additional user requires real compute, power, and silicon.
Generating a single token costs roughly (model parameters) x (context length squared) in FLOPs — orders of magnitude more compute than any prior paradigm like database retrieval.
Groq’s deterministic dataflow architecture with on-chip SRAM is over 10x faster on memory-bandwidth-bound operations than GPU HBM, making it ideal for the decode phase of inference.
Disaggregating inference — splitting prefill from decode, then further splitting compute-intensive and memory-bandwidth-intensive operations within decode — yields 2.5x more tokens per watt when combining Groq and Nvidia chips.
Nvidia acquired Groq for $20B in roughly 30 days from first working prototype to deal. Complementary architecture (not a better GPU) made cultural integration feasible.
Inference cost dropped 99% in 2.5 years, but model sizes are approaching 1-10 trillion parameters, so hardware gains are continuously consumed by larger models and growing demand.
Three inputs drive inference cost: semiconductor supply chain (TSMC, lithography), engineering innovation (chip design, quantization), and available power.
Anthropic went from $3.5B to $10.5B annualized revenue in a single quarter (Jan-Mar 2026), driven purely by product capability crossing a threshold — no salesforce expansion.
OpenAI and Anthropic started with negative gross margins (selling a dollar of compute for 20 cents). Now at positive gross margins as inference costs fell and willingness to pay rose.
Reasoning models and agents consume vastly more tokens than chat, but deliver proportionally more value — shifting the unit economics favorably.
Current frontier models haven’t even been trained on latest hardware (Blackwell, Vera, Rubin). Capabilities will step-change when they are.
Gerstner’s thesis: IQ gets commoditized, EQ (persuasion, leadership, network) becomes the scarce valuable skill.
An 8B parameter model (tiny by current standards) drains an iPhone in 30 minutes. Running frontier intelligence on the edge remains impractical.
Jensen Huang’s internal bar: every new generation must deliver 100x improvement, not incremental gains.
Nvidia has $1 trillion in booked sales over the next 8 quarters, with demand still exceeding available memory and supply.

Claude’s Take

This is a solid insider’s view of AI infrastructure economics, delivered by people with real skin in the game — Gerstner manages $15B and holds major positions in Nvidia, OpenAI, and Anthropic; Madra literally built Groq Cloud and brokered the Nvidia acquisition. That proximity to the deals gives the discussion a specificity you don’t get from most AI commentary. The Anthropic revenue figures ($3.5B to $10.5B in three months) and the Nvidia acquisition timeline (prototype to $20B deal in 30 days) are the kind of concrete data points that cut through the noise.

The weaknesses are predictable for the format. This is a fireside chat at a Stanford class, not an adversarial interview. Nobody pushes back on whether Gerstner’s Nvidia price target is just his book talking, or whether Anthropic’s revenue spike is sustainable versus a one-time adoption wave. The “AI bubble” question gets treated as definitively settled by one quarter of revenue data, which is a bit fast. And the safety discussion is thin — the Mythos sandboxing gets a couple of minutes before pivoting back to optimism.

The technical content on inference disaggregation is genuinely useful and clearly explained. The economics framing — negative gross margins flipping positive as cost curves and capability curves crossed — is the cleanest articulation of the AI business model I’ve seen from the investor side. The “three-dimensional cube” metaphor for the inference cost problem (demand, model size, and hardware all growing simultaneously) is a good mental model even if it’s not exactly rigorous.

Score: 7/10. High-quality primary source material from principals directly involved in the deals they’re discussing. Docked for the lack of pushback, the heavy Nvidia/Groq promotional angle (both speakers have direct financial interest), and the somewhat shallow treatment of safety and distribution problems. Still, the concrete numbers and insider perspective make this worth the time.