5 Papers That Show Where AI Research Is Heading Right Now

ELI5/TLDR

Five young researchers each grabbed a recent AI paper and explained it to a room of fellow nerds. The thread running through all of them: the recipe that made ChatGPT good — throw more data and compute at a simple task — is now spreading into biology, math, and software. Proteins are being treated like language. Models are being taught to invent their own homework. Math proofs are being checked by an unfoolable machine. And one founder closes by arguing you should run your coding agents like a real-time strategy game. It is a snapshot of where the frontier is poking, not a polished documentary.

The Full Story

This is a meetup, not a lecture. A guy named Francois runs an AI paper club and opens by riffing on what obsesses him — memory, learning more from each example, and the gap between copying humans and surpassing them. Then five people present. The quality is uneven and the audio is a transcript of people talking fast, but the ideas are real. Here is what each one is actually about.

The bitter lesson comes for biology

The first speaker, a PhD student working on AI-for-biology, sets up the whole session with one idea worth holding onto: the bitter lesson. It comes from a famous 2019 essay by AI researcher Richard Sutton. The claim is uncomfortable. Across 70 years of AI, the methods that win in the long run are almost never the clever ones where humans hand-build their knowledge into the system. The winners are the dumb, general methods that just eat more compute and more data. Hand-crafted cleverness wins early, then gets overtaken and left in the dust.

His paper asks: does this hold for proteins too?

A quick chaperone for the biology. A protein is just a string of beads, where each bead is one of 20 possible amino acids. Think of it as a 20-letter alphabet. The order of the letters decides how the string folds into a 3D shape, and the shape decides the job — fighting off a virus, speeding up a chemical reaction, and so on. For decades, reading protein structure was painstaking human work.

The trick the researchers borrow from language is this: take the “you shall know a word by the company it keeps” idea — you can guess a missing word from its neighbors — and apply it to proteins. Hide some amino acids, train a big model to guess them from the ones nearby, and never tell it anything else. No structures, no labels, just sequences. The bet is that if you do this at enormous scale, the model quietly learns deep facts about proteins as a side effect.

“You’ll know a word by the company that it keeps and here the idea is that you’ll know a protein by amino acids it keeps.”

And it works. The new model family (ESM Cambrian) keeps improving smoothly as you add compute and data — the same clean log-linear curve you see in language models. The previous generation had hit a wall and flattened out. The fix was not a clever new architecture. It was simply more data: they went from 50 million training sequences to 2.8 billion, much of it scraped from DNA found in dirt, oceans, and human guts — organisms nobody has ever cultured.

The punchline that should stick: evolution has been generating protein “training data” for four billion years, and we have so far sampled less than 1% of known protein diversity. In language, people worry about running out of text. In biology, there is no data wall in sight.

The most striking result: a model trained purely on this fill-in-the-blank game, with no hand-built features, nearly matches AlphaFold (the Nobel-winning structure predictor) — and actually beats it on antibody design, which is exactly the high-value task where the old hand-built shortcuts run dry. And when researchers crack open the model’s internals, they find it has spontaneously organized its knowledge into a clean hierarchy of real biological concepts — individual amino acids, then structural motifs, then whole functional roles — without anyone teaching it the categories.

Teaching a model to write its own homework

The second talk, by a PhD student named Luke Bailey, is about self-play — the trick behind AlphaGo, now being aimed at large language models.

Start with the problem. The current way to make models smarter after pre-training is reinforcement learning: collect a huge pile of tasks (coding problems, math problems), let the model attempt them, reward the good attempts, punish the bad ones. It works, and it scales beautifully — more tasks, more compute, reliably better model. But there is a catch. Humans have to hand-collect all those tasks. Eventually you want the model to outgrow any problem a human can pose.

Self-play is the proposed escape. The model plays two roles. One role — call it the conjecturer — invents new problems. The other role — the solver — tries to solve them. You train both to get better. In AlphaGo’s version, the “problem” was just an older copy of the agent to play against. The newer version is looser: one part of the model dreams up entire tasks, the other part attacks them. The dream is a machine that generates its own endless curriculum and improves forever.

“In principle nothing bounds learning… selfplay on the other hand is going to say I’m going to keep on generating new learning signal with new tasks, learn it and just keep on improving hopefully forever. But unfortunately, this is not how it works.”

Here is where it gets honest. Run naive self-play and it plateaus, exactly like plain RL. No magic. The paper digs into why, and the diagnosis is funny and a little dark. The conjecturer is rewarded for producing problems the solver finds hard. But the laziest way to make a problem hard is to make it ugly and convoluted, not deep. Imagine asking someone to set you a “50%-pass-rate” exam and they just hand you a three-page calculus problem where you’ll slip up somewhere. Technically hard. Completely useless for learning anything real.

Their fix (called self-guided self-play): anchor the invented problems to real ones you couldn’t solve, and add a third role — a guide that judges whether the new problem is genuinely related and not just artificially complex. The result: a small 7-billion-parameter model, given 8x the compute, claws its way up to the ability of a model nearly a hundred times its size. A real result, but the speaker is refreshingly clear that it is one step, not a solution.

Running RAG before you finish the sentence

The third talk, by a researcher at the fast-growing startup Giga, is the most applied and the lightest. It is about voice AI and a problem called latency.

Background chaperone: RAG (retrieval-augmented generation) is the standard trick to stop a model from making things up. Before answering, you go fetch relevant documents and hand them to the model so it answers from real sources. The problem in voice: fetching takes time, and a 10-second pause before a voice agent replies feels broken. Humans also catch fabrications far worse by ear than by eye, so getting it right matters more in voice.

The clever idea in the paper: don’t wait for the user to finish speaking. Start running retrieval on the words as they arrive. If someone asks “what’s the weather today, I’m deciding whether to go out” — the useful part is the first half. So the system watches the incoming chunks and tries to decide, mid-sentence, when it has heard enough to go fetch. The speaker is honest that the specific method (comparing whether the documents retrieved from a partial question match those from the full question) is just one approach, and that the real value is in flagging an open research problem. The payoff in the paper: roughly 0.5 to 1.5 seconds shaved off latency with no loss of accuracy.

Math that cannot be faked

The fourth talk, by a Caltech PhD student named Robert George, is about Lean and what he calls verified intelligence.

The setup: informal math — the kind you did in school — is flexible and forgiving. Steps get skipped, “proof by intimidation” happens, and you mostly trust the human. Lean is the opposite. It is a programming language for writing math where every single step must be spelled out, and a checker (the “kernel”) verifies it. You cannot hand-wave. You cannot fool it. If Lean says your proof is correct, it is correct, full stop.

Why this matters now: AI models are getting good at math, but you can’t always trust their reasoning. Pair them with Lean and you get the best of both — the model’s creativity plus an unfoolable referee. The speaker rattles off a wave of recent breakthroughs: models solving decades-old open problems, gold medals at the International Math Olympiad, AI proofs verified end-to-end in Lean. He extends the idea past math into two other domains: code that needs guarantees (proving a program actually does what its spec says, not just that it compiles) and science (reproducibility). He even built a framework to write neural networks inside Lean and prove properties about them — for instance, formally proving that an optimized attention mechanism is equivalent to the standard one. The pitch: shift from “vibe coding” to “verifiable coding.”

Programming like a real-time strategy game

The last talk is the antithesis of the fourth — no proofs, maximum chaos. Luke Orthwine, a founder running a consumer AI company, argues that coding with AI agents should feel less like chess and more like a real-time strategy game (think StarCraft or Warcraft).

His logic: old-school programming was linear and single-threaded. You designed carefully, you worked on one thing at a time, you aimed for correct. Agentic coding is the opposite. You are commanding many units (agents) at once across a map, none of them perfect, all of them needing the occasional correction. You can’t win an RTS by perfectly controlling one unit while you forget to build an army. So you optimize for throughput and parallelism, not per-unit perfection.

The concrete practices, stripped of the gaming metaphor:

Spawn lots of agents in parallel, each told to push as far as it can before asking for help — even if it’s sometimes wrong, because correcting later is cheap and your time is the scarce resource.
Keep high visibility. Don’t fire off 20 agents and pray; constantly glance between them and course-correct early.
Document aggressively into linked “knowledge base” files, because code is an expensive thing for an agent to read context from, and good docs make future agents faster.
He literally maps each agent to a Warcraft/StarCraft unit with its sound effects, so an audio cue tells him which agent needs attention — the same way pro players manage an army by ear.
His team built an “APM tracker” (actions per minute, borrowed from RTS) measuring tool calls per minute. High activity isn’t sufficient to be good, but nobody good has low activity.

He claims this raised his team’s output to 3.5x the PRs per engineer per month, with another 60% jump after the whole team adopted it. Take the numbers with salt — this is a founder selling a workflow — but the underlying observation (the bottleneck has moved from typing code to orchestrating attention) is genuine.

Key Takeaways

The bitter lesson (Richard Sutton, 2019): over decades, general methods that scale compute and data beat methods that hand-build human knowledge. Hand-crafted cleverness wins early, then loses.
A protein is a string from a 20-letter alphabet (amino acids); the sequence determines the 3D fold, and the fold determines the protein’s job.
Masked-language pre-training on raw protein sequences — hide some amino acids, predict them — makes a model learn structure and function as a side effect, with zero structural labels.
Scaling protein models worked not via a clever architecture but by pushing training data from 50M to 2.8B sequences, much of it from uncultured organisms in dirt, oceans, and guts.
We have sampled less than 1% of known protein diversity. Unlike text, biology has no visible data wall — evolution has generated data for 4 billion years.
A pure sequence model nearly matches AlphaFold and beats it on antibody design — the exact task where the old hand-built shortcut (multiple sequence alignment) is scarce.
Protein models spontaneously organize their internal features into a human-readable hierarchy of biological concepts, found via sparse-autoencoder interpretability tools borrowed from language-model research.
Self-play for LLMs: one model role invents tasks (conjecturer), another solves them (solver), both trained to improve — aiming for a self-generating, endless curriculum.
Naive self-play plateaus because “make it hard for the solver” is gamed by producing convoluted ugly problems, not deep ones. The fix anchors generated problems to real unsolved ones and adds a “guide” role to judge relevance.
Self-guided self-play got a 7B model (with 8x compute) up to the ability of a model ~100x larger — a step, not a solved problem.
Streaming RAG: run retrieval on a voice query while the user is still speaking to cut the latency that makes voice agents feel broken; the open problem is deciding mid-sentence when you’ve heard enough.
Lean is a language where every proof step must be explicit and a kernel checks it — you cannot fool it. Pairing AI creativity with Lean’s unfoolable verification gives “verified intelligence.”
The same verification idea extends to code (prove a program meets its spec) and science (reproducibility); “vibe coding” → “verifiable coding.”
Agentic coding resembles real-time strategy more than chess: optimize throughput and parallel attention over per-unit perfection; let agents run far before asking for help; course-correct early with high visibility.

Claude’s Take

This is a paper-club recording, and it shows. The audio is rough, the transcription mangles names and terms (“ESM Cambrian” comes through as “ESM cranberry”), and nobody is performing for a camera. That’s actually a feature — you’re hearing what working researchers are genuinely excited about, unvarnished, rather than a polished thought-leadership take.

The connective tissue is real and worth the time: the bitter lesson is the load-bearing idea across the first two talks, and watching it generalize from language to proteins to self-generated curricula is the actual signal here. The protein talk is the strongest — clear, honest about where the method still falls short, and genuinely surprising (a fill-in-the-blank model out-designing the Nobel-winning system on the task that matters most). The self-play talk is the most intellectually honest, openly admitting the approach plateaus and that they took one step, not a victory lap.

The two weaker entries drag the average. The streaming-RAG talk is fine but thin — the speaker himself keeps saying the method isn’t the point. The RTS-coding finale is entertaining and contains a real insight (the bottleneck has moved from writing code to directing attention), but it’s a founder pitching a workflow with self-reported productivity numbers and a heavy gaming aesthetic that won’t survive contact with most teams. “Run everything in dangerously-skip-permissions mode” is advice that works until it spectacularly doesn’t.

Score: 7. High signal in the first half, a genuinely useful mental map of where the frontier is poking, but uneven quality and a recording that demands patience. Worth it if you want the texture of current AI research; skippable if you want a clean narrative.