Stanford Just Revealed ChatGPT's Secret | Full Breakdown

ELI5/TLDR

A Stanford lecture walking through how a chatbot like ChatGPT actually gets built, end to end. The short version is that the headline ingredient — the famous transformer architecture — turns out to matter less than three boring things almost no one writes about: what data you feed it, how you measure if it works, and how you wring every drop out of the GPUs. The clever stuff happens in two stages: first you train a model to imitate the entire internet, then you spend a tiny extra round teaching it to behave like a helpful assistant.

The Full Story

What actually matters when you build one of these things

The lecturer opens with a list of five ingredients and immediately admits academia gets the priorities wrong. Architecture and training algorithm are what professors love writing papers about. In practice, what makes or breaks a real model is data, evaluation, and systems — the unglamorous middle of the production line. He spends the lecture mostly there, skipping past the transformer architecture entirely.

The whole field splits into two phases. Pre-training is the classical move: dump the entire internet into a model and have it learn to predict the next word. Post-training is the recent twist: take that finished model and bend it into something that can hold a conversation. GPT-3 was pure pre-training. ChatGPT was the first one that got serious about the second phase. That phase, more than anything else, is why the public suddenly cared.

Pre-training in one sentence

A language model is just a probability machine for sequences of words. Given “the mouse ate the,” it assigns a probability to “cheese.” Multiply those one-word-at-a-time probabilities together and you’ve modeled a sentence. The training trick is just classification — at every position, predict the next token, compare to what actually came next, nudge the weights. Cross-entropy loss. The same loss undergrads see in week three of a machine learning class. Nothing exotic.

Tokens — the strange middle layer between text and math

Words aren’t the unit of training. Tokens are. A tokenizer chops text into chunks roughly three or four characters long. Why not just use whole words? Because typos break it, and languages like Thai don’t use spaces. Why not just use single characters? Because then your sequences get so long the model crawls — transformers get quadratically slower as input length grows.

The trick of choice is byte-pair encoding. Start with every character as its own token, then walk through a giant pile of text and merge the most common pair into a new token. Repeat. Eventually “token” itself becomes a token, and “tokenizer” becomes “token” + “izer.”

This sounds boring but it has weird downstream effects. Numbers like 327 get crammed into a single token, which is part of why language models are so terrible at arithmetic — they literally don’t see digits the way you do. GPT-4 made a big jump on coding partly because they fixed how Python’s four-space indents were being tokenized. The thing that academics dismiss as plumbing turns out to control what the model can think about.

Perplexity — the score nobody publishes anymore

The classical way to measure a language model is perplexity. Take the model’s average per-word loss, exponentiate it, and you get a number between one and the size of the vocabulary. Think of it as: how many words is the model hesitating between at each step? A perfect model hesitates between one. A model with no clue hesitates between all of them. Between 2017 and 2023, perplexity on a standard benchmark dropped from about 70 down to under 10.

Nobody uses perplexity for cross-model comparisons anymore because it depends on the tokenizer — a model with a 100,000-token vocabulary gets compared on a different scale than one with 10,000. Instead, academia uses big bundles of question-and-answer benchmarks like MMLU. The MMLU trick is sneaky: rather than asking the model to write an answer, you give it four choices and check which one the model thinks is most likely. No need to grade essays.

The data pipeline is a hidden universe

The lecturer keeps stressing that “trained on the internet” is doing enormous work. What actually happens:

Step one, download the internet. Common Crawl ships about 250 billion pages a month, roughly a petabyte of HTML. A random page from that pile is unreadable garbage — broken sentences, shopping cart UI, half-rendered footers.

“Test king world is your ultimate source for the system X high performance server… and then you have three dots so you don’t even — the sentence is not even finished. That’s how random internet looks like.”

Then comes a long parade of cleaning steps, each of which is its own engineering problem. Strip the HTML. Yank out unsafe content. Deduplicate — same paragraph can appear thousands of times across the web. Filter for quality with simple rules (suspiciously long words, suspiciously short pages). Then a clever model-based filter: train a small classifier to recognize text that looks like the kind of stuff Wikipedia would link to, and upweight it. Sort what’s left into domains — code, books, entertainment — and dial each one up or down. Up-weighting code, oddly, seems to make models better at general reasoning.

At the very end of training, you anneal — drop the learning rate to nearly zero and overfit on a small pile of really high-quality stuff like Wikipedia. The model gets one last polish.

How big is the final cleaned dataset? Llama 3 was trained on 15 trillion tokens. That’s roughly the high-water mark anyone has admitted to publicly. GPT-4 is probably similar. About fifteen people on a seventy-person team work on data alone.

Scaling laws — the strangest empirical fact in the field

Around 2020, OpenAI noticed something that violated everything anyone teaches in a regular machine learning class: bigger models trained on more data just keep getting better. No overfitting. No plateau. Plot test loss against compute on a log-log scale and you get a straight line. Same for data. Same for parameters.

“There’s no empirical evidence of plateauing anytime soon. Why? We don’t know.”

This sounds innocuous. It is in fact the load-bearing fact that the entire AI industry is built on. If the line keeps going down, you can predict in advance how much better next year’s model will be just by knowing how many GPUs you bought.

Scaling laws turned model development from a guessing game into something closer to engineering. Old way: you had 30 days of compute, so you trained 30 different small models for a day each and picked the best one. New way: you train tiny models at several different sizes, fit a curve, extrapolate, then spend 27 of your 30 days training one giant model you’re confident about. This is also how you compare architectures honestly — fit a curve for each, see which line goes down faster.

The Chinchilla paper from DeepMind nailed down the optimal trade-off: about 20 tokens of training data per model parameter is the sweet spot for training cost alone. But once you also factor in the cost of running the model after it’s built, you want a smaller model trained on more data — closer to 150 tokens per parameter. Most production models live near that ratio.

Richard Sutton’s bitter lesson

If compute keeps getting cheaper and bigger models keep getting better, then the only thing that matters long-term is: build architectures that can use compute. Don’t waste your life inventing clever tricks that improve the intercept of the curve by 2%. Just scale.

“Don’t spend time overcomplicating. Do the simple things. Do it well. Scale them. That’s really what OpenAI taught us.”

The lecturer notes ruefully that most academic researchers, himself included, spent years on exactly the wrong things.

What it actually costs to train one of these

Llama 3, the 405-billion-parameter version, ran on 16,000 H100 GPUs for about 70 days. Roughly 26 million GPU-hours, which at $2 an hour is about $52 million in rental cost. Add fifty engineers at half a million each per year — call it $25 million more. Total: roughly $75 million. Carbon footprint: about 4,400 tons of CO2, equivalent to 2,000 round-trip flights JFK to London. Each new generation of frontier model uses roughly 10x more compute than the last.

There’s also a US executive order from the Biden administration that triggers extra government scrutiny once a model crosses 10^26 floating point operations. Llama 3 came in at 3.8 × 10^25 — almost suspiciously close to the limit, almost certainly on purpose.

Post-training — turning a giant text predictor into a helpful assistant

A pure language model isn’t an assistant. If you ask GPT-3 “explain the moon landing to a six-year-old,” it might respond by listing more questions in the same format — because that’s what the internet looks like, lists of questions. To make a chatbot, you have to teach the model that one specific style of response, the helpful answer, is the one you want.

The old fix is supervised fine-tuning. Pay humans to write good answers, then keep training the model on that small pile of examples using the same loss function as before. The surprise is how little data this takes — going from 2,000 examples to 32,000 barely moves the needle. The reason is that the model already knows everything; you’re just telling it which of its existing personas to lean into.

“All you tell the model is to specialize on one type of user that it saw already in the pre-trained data set. So the knowledge is already in the pre-trained LLM.”

Stanford’s Alpaca project showed you can skip the human writers entirely for this stage. Have a stronger model generate the training pairs, then fine-tune a weaker model on those. It works embarrassingly well.

RLHF — preferences instead of clones

Supervised fine-tuning has a sneakier problem: it teaches the model to imitate humans, but humans aren’t always good at producing what they want to consume. You can recognize a great novel without being able to write one. Worse, if a human writes an answer that references a book the model has never heard of, the model learns to fabricate plausible references. Hallucination, the lecturer suggests, may be partly an artifact of forcing the model to confidently produce text it has no reason to believe.

The fix is reinforcement learning from human feedback. Have the model generate two answers. Show them to a human. Ask which is better. Repeat thousands of times. Then nudge the model toward the kind of thing humans pick.

The original recipe, used by ChatGPT, runs in three steps: supervised fine-tune, train a separate “reward model” to predict human preference scores, then use a reinforcement learning algorithm called PPO to crank the language model toward higher rewards. PPO works but is famously messy — clipping, rollouts, hyperparameter babysitting, the whole reinforcement learning circus.

A Stanford paper from a year ago proposed DPO, which is just: directly maximize the probability of the answers humans liked and minimize the probability of the ones they didn’t. Same outcome, much simpler implementation, no reward model needed. It’s now the open-source standard.

The data labeling part has its own pathologies. Humans agree with each other only about 66% of the time on which answer is better — even five paper authors who’d spent hours discussing the labeling guidelines couldn’t push past 68%. They tend to vote for whichever answer is longer, even when length isn’t relevant. RLHF inherits this bias, which is why ChatGPT’s answers keep getting wordier with each model generation. Replacing human labelers with strong LLMs is now common: about 50x cheaper, and somewhat more consistent because the models lack the human variance.

Evaluating chatbots is its own nightmare

Once you’ve moved past pre-training, you can’t even use perplexity anymore — the model after RLHF isn’t really modeling a probability distribution, it’s a policy trying to maximize one specific reward. The current gold standard is Chatbot Arena: random users on the internet talk blindly to two chatbots and vote. Aggregate over hundreds of thousands of votes and you get a ranking. Slow and expensive, but trustworthy. Cheaper alternative: have GPT-4 do the judging for you. The Alpaca Eval benchmark using LLM judges correlates 98% with Chatbot Arena and runs in three minutes.

But the LLM judges share the human bias toward longer outputs. If you simply tell GPT-4 to “be verbose” in the prompt, it wins 64% of head-to-heads against itself. “Be concise” drops it to 20%. Length is doing a lot of the work that everyone is calling quality.

Why GPUs matter, briefly

The last few minutes are a sprint through systems. GPUs are built for throughput, not latency — many cores doing the same operation on different data. They’re optimized for matrix multiplication, which is part of why the entire field looks the way it does: anything you can phrase as a matrix multiply runs about ten times faster than anything else.

The other constant headache is communication. Compute has gotten faster much quicker than memory bandwidth. Even at companies like Meta, the model’s actual GPU utilization sits around 45% — most of the time the chips are idle, waiting for data. Two of the simplest tricks: use 16-bit floats instead of 32-bit for the actual math (the noise from training drowns out the precision loss), and “fuse operations” so you don’t shuttle data back and forth between memory and compute for every line of PyTorch. Just running torch.compile on a model gets you roughly a 2x speedup for free.

Key Takeaways

Architecture matters less than data, evaluation, and systems. Most of academia works on the wrong end of the problem.
Tokenizers are the silent layer that controls what a model can do. Bad tokenization of numbers is a major reason LLMs fail at arithmetic.
Perplexity is “how many words is the model hesitating between” — went from ~70 to under 10 between 2017 and 2023.
“Trained on the internet” actually means a multi-stage pipeline: crawl, dedupe, filter for quality (Wikipedia-link classifier is a clever trick), classify by domain, upweight code/books, anneal on Wikipedia at the end.
Scaling laws: bigger model + more data = predictable improvement on a log-log line. No plateau yet. This is the empirical fact the entire industry runs on.
Chinchilla rule: ~20 training tokens per parameter is optimal for pure training cost. ~150 tokens per parameter once you also account for inference cost.
Bitter lesson: don’t invent clever architectures, build things that scale. The intercept of your curve doesn’t matter, only the slope.
Llama 3 405B cost roughly $75M to train (compute + salaries) in 70 days on 16,000 H100 GPUs. Each generation uses ~10x more compute.
Pre-training teaches a model the internet. Post-training teaches it to be an assistant. ChatGPT was the breakthrough of step two.
Supervised fine-tuning needs surprisingly little data (~2,000 examples) because the model isn’t learning, it’s choosing which voice to use.
RLHF teaches the model human preferences instead of cloning behavior. May reduce hallucination from cases where the model didn’t know an answer was true.
DPO replaces the messy PPO reinforcement-learning pipeline with simple maximum-likelihood training. Same outcome, much less plumbing.
Humans only agree 66% of the time on which chatbot answer is better. They vote for longer answers. RLHF amplifies this — every generation of ChatGPT gets wordier.
LLM judges (Alpaca Eval) correlate 98% with human preference rankings, ~50x cheaper than humans.
GPUs sit ~55% idle even at Meta because compute outruns memory bandwidth. Two cheap wins: use 16-bit floats, and run torch.compile to fuse operations.
A simple flop count formula: roughly 6 × parameters × tokens. Useful for back-of-envelope estimates of training cost.

Claude’s Take

The title is borderline clickbait — there’s no “secret” being revealed, this is a regular Stanford lecture by what sounds like Yann Dubois (one of the Alpaca authors). But the channel uploaded the actual lecture more or less intact, which is the only thing that matters. The content is genuinely good — a working researcher walking through the entire LLM stack, with the right kind of hand-waving in the right places and the right asides about what actually matters in practice versus what gets published.

What lifts this above a generic “intro to LLMs” video is the lecturer’s repeated insistence that the field’s prestige hierarchy is upside down. Everyone wants to invent new architectures; almost nobody wants to clean training data; the people cleaning training data are the ones building working products. The bitter lesson framing — that scale beats cleverness — is a few years old now but still poorly internalized. He’s also generous with concrete numbers ($75M to train Llama 3, 16,000 GPUs, 70 days, 4,400 tons of CO2) which is rare in this kind of overview.

Worth noting where he stops being neutral: as a coauthor on Alpaca, Alpaca Farm, Alpaca Eval, and DPO, he naturally talks up the Stanford-flavored simplifications of OpenAI’s recipes. That’s not wrong — DPO really did become the open-source standard — but you’re getting the Stanford camp’s view on which ideas mattered. Score: 8/10. Not life-changing if you already work on this, but if you want one lecture that demystifies “how does ChatGPT actually get built,” this is one of the cleanest you’ll find.