A Multiscale Logic of Collective Intelligence - Donald Hoffman and Chetan Prakash
A Multiscale Logic of Collective Intelligence - Hoffman and Prakash
ELI5/TLDR
Donald Hoffman thinks space, time, and matter aren’t the bottom floor of reality. He thinks conscious experiences are — and everything else, including your body, your planet, and quantum physics, is built on top of them. In this talk he sketches a surprisingly simple bit of math (basically a chain of “what am I seeing now, what do I see next?”) that he claims can generate agency, goals, intelligence, and eventually even Einstein’s relativity. Michael Levin, the biologist who makes flatworms grow two heads by zapping them with electric fields, is on the call nodding along because it might explain where that weird biological intelligence comes from.
The Full Story
The setup: two guys in a Zoom call who think physics has been doing it wrong
Picture a small online meeting. On one side: Donald Hoffman, the cognitive scientist famous for arguing that what you see is not what’s really out there — that perception is more like a desktop interface than a window onto reality. On the other side: Michael Levin, the biologist who studies how embryos and cells seem to “know” what to build even when you cut them in half. Also present: Chris Fields, a physicist-philosopher, and Robert Prentner, a consciousness researcher who trained with basically everyone — Tononi, Chalmers, Anil Seth.
They are here because Hoffman has a new piece of math he wants to run past them. He calls it the recursive trace logic. He’s visibly excited about it. He says it’s only two months old.
Why start from scratch?
Hoffman opens with a strange claim: space-time is doomed. And not his claim — he’s quoting Nima Arkani-Hamed, a high-energy physicist at Princeton’s Institute for Advanced Study. The European Research Council has just handed ten million euros to a hundred-plus physicists and mathematicians to study what they call positive geometries — mathematical objects that spit out the answers physics experiments get, without ever mentioning space, time, or quantum theory.
“Space-time is doomed. There’s no such thing as space-time fundamentally in the actual underlying description of the laws of physics.” — Nima Arkani-Hamed
The reason, Hoffman explains, is that when you try to mash together Einstein’s general relativity with quantum theory, space-time stops making sense at extremely tiny scales — something like a trillionth of a trillionth of a trillionth of a centimeter. Smaller than that, the very notions of “here” and “there” and “before” and “after” don’t mean anything anymore. So if you take physics seriously, you need a deeper foundation. Space-time is just a useful approximation — the way Isaac Newton’s physics is still useful for throwing baseballs even though we know Einstein’s version is more correct.
This matters for consciousness research because almost every theory of consciousness starts by asking “what arrangement of stuff inside space-time could give rise to conscious experience?” If space-time itself is built on something deeper, that whole project might be looking under the wrong streetlight.
Hoffman’s move: start somewhere else entirely. Start with experiences.
The world’s simplest mind
Here’s where the math begins, and it begins almost insultingly simple.
Imagine a creature whose entire mental life consists of four experiences: red, green, blue, and yellow. Right now it’s seeing red. A moment later, green. Then yellow. Then green again. That’s the whole creature. It just has these flickers of experience that change.
To describe how one experience leads to the next, you use a thing called a Markov chain. Don’t let the name scare you — it’s just a table of probabilities. If I’m currently seeing red, there’s a 20% chance I’ll see red next, a 30% chance I’ll see green, a 30% chance blue, and a 20% chance yellow. You write this out as a grid, the grid is the “matrix,” and that grid is the whole description of the creature’s inner life.
Here’s the first quietly interesting thing about Markov chains: they have goals baked in without anyone programming them. No matter where you start the chain, it tends to drift toward a specific long-run pattern — a kind of attractor. Perturb it, push it around, and it resists and drifts back. William James once defined intelligence as “achieving a fixed goal with variable means.” By that definition, even this ridiculously simple color-flicker creature is already a little bit intelligent. It has a destination.
Traces: what the small mind sees of the big world
Now Hoffman introduces scale. Imagine a second, simpler observer who can only see two colors — red and green. The “real” dynamics involves four colors bouncing around, but this observer only catches red and green. What transition probabilities does this smaller observer experience?
You can’t just copy the red-and-green corner of the original table. The smaller observer’s dynamics are actually affected by all the hidden detours through blue and yellow. If being red leads to being blue leads to being green, the two-color observer sees a direct red-to-green jump, but it’s happening at a rate that depends on all that invisible machinery.
There’s a formula for computing this — it’s called the trace of the Markov chain. Mathematicians have known about it for over half a century. Hoffman isn’t inventing it. The formula basically says: start with the part you can see, then add a correction that sums up every possible path the system could take through the hidden states before popping back into view.
You can think of it like watching a reality TV show but only seeing the confession booth. You see a contestant happy, then sad, then happy again. The trace formula tells you the statistics of those mood swings even though the actual cause — everything that happened outside the booth — is invisible to you. The hidden stuff still shapes what you see.
Hoffman names the pieces. A is the visible states. C is the dark dynamics you can’t see. B is the exits from visible to invisible. D is the re-entrances back. Those hidden layers — B, C, D — end up acting like memory and control, influencing what the small observer experiences without the small observer ever detecting them directly.
The new trick: trace logic is a logic
Here’s what Hoffman and Prakash discovered a couple years ago. If you line up all possible Markov chains and ask “is chain M a trace of chain N?” you get a partial ordering — a way of saying some chains are smaller, simpler views of bigger ones. And partial orderings give you logic. Not Boolean logic like the AND/OR/NOT gates in a computer chip, but something more exotic with its own rules about when things combine and when they don’t.
Hoffman calls this the trace logic, and he claims it’s the logic of minimal surprise. The idea: if you’re a small observer looking at a big system, the trace is the most “un-surprised” view you can have of the big system — it’s what you expect given only what you can see, with nothing extra assumed. Karl Friston fans will recognize this as deeply related to the free energy principle, the idea that brains minimize surprise. Hoffman is basically saying: here’s the mathematical scaffolding for doing that across many scales at once.
Agency: choosing which window to look through
Now the new part. The thing that’s only two months old.
Hoffman asks: what is agency? What does it mean for a thing to act?
His answer is strange and elegant. Agency is choosing which observer window to look through next. If you’re the two-color creature and you could shift to being a three-color creature, or a different two-color creature, that shift is an act. And the rule you follow for deciding which window to shift to — that’s a policy.
What is a policy, mathematically? It’s another Markov chain. A chain whose states are observer windows and whose transitions tell you the probability of switching from one window to another.
And since policies are themselves Markov chains, they have their own trace logic. So you can have policies for choosing policies — meta-policies. And meta-meta-policies. And so on, as deep as you want to go. Hoffman calls this recursive trace logic, and it’s the new observation that got him excited enough to call Levin’s group.
There’s something wonderfully austere about this. One definition (the trace). One observation (it makes a logic). One further observation (the logic itself supports more Markov chains, which means it’s recursive). From those three moves he’s trying to grow a theory of agents, selves, pleasure and pain, minds, and eventually physics.
“What I love about it is it’s austere. There’s only one equation and one logic… And yet the claim is we should be able to get everything out of it.”
Wait, what does “moving my hand” mean in this picture?
Chris Fields spots the oddity and asks. If the only action available to an agent is “choose which experience to have next,” what does it mean to say you moved your hand?
Hoffman’s answer is the part that requires really putting the old furniture in storage:
“I have an observer window where my hand is touching my ear. Now I want an observer window in which my hand is touching my leg. And so I transition to that observer window. What’s happening is I’m choosing what I want to see in my movie next. And that’s what we call moving my hand.”
Read that twice. It’s saying the body isn’t moving in an external world. The agent is scrolling through a menu of possible experiences and picking the next one. The sensation we call “motor action” is just a particular kind of transition in that menu. The hand never moves because there is no hand in the basement-level description. There are just experiences and the transitions between them.
Building a “self” without a Markov blanket
If there’s no external world, what distinguishes me from the rest of reality? Standard theories use something called a Markov blanket — a clean mathematical boundary between an agent and its environment. But Markov blankets only work for a specific kind of simple graph, and Hoffman’s chains are more general. So he’s inventing a replacement he calls a trace blanket.
The self, in this picture, is something an agent learns by noticing patterns. My hands and my body show up in almost every observer window I have. My phone doesn’t. My ability to directly control hand-experiences is high; my ability to control phone-experiences is low (I have to go through a hand-move-phone sequence). Certain experiences come attached to pleasure signals, others to pain. Over time, I build up a statistical picture of “what always travels with me” versus “what’s contingent.” That cluster becomes the self. It’s not given. It’s inferred from the flicker of experiences.
The intelligence dial
Hoffman points out something useful. For any “goal” — meaning any particular long-run distribution a Markov chain drifts toward — there are actually infinitely many chains with the same goal that get there at different speeds. Some arrive in two steps no matter where they start. Some take forever. The quantity that determines speed is a specific number called lambda two — technically, the second-largest eigenvalue of the matrix. (If you’re lost on eigenvalues: think of lambda two as the “stubbornness” of the chain, how hard it resists being pulled to its goal. Smaller lambda two means quicker surrender to the attractor.)
This gives you a way to measure intelligence: how efficiently does the chain find its goal? Mike Levin’s lab has their own intelligence metric called K, which measures search efficiency. Hoffman notes that lambda two and K are mathematically linked.
And the reason Hoffman cares is that higher layers of the recursive policy stack can bend how the lower layers search — not metaphorically but literally, by changing which states cluster together into communities. Think of it like this: if you want to find something in a messy room, one kind of intelligence is “search the whole room fast.” Another, deeper kind is “reorganize the room into zones so you almost never have to search it.” Community structures in the Markov chain are like zones, and meta-policies can carve them.
And now, somehow, relativity
This is where Hoffman leans forward and makes his biggest claim. He thinks the trace logic can produce special and general relativity from scratch — no physics assumed, just agents watching experiences flicker.
The trick involves giving each observer a ticking counter. Every time your experience changes, the counter bumps up by one. That’s your clock. Here’s the elegant part: a sub-observer (one who only sees a subset of the big observer’s states) has a slower counter, because it only notices some of the transitions. So being a smaller, coarser observer means your time runs slower relative to a finer observer.
That, Hoffman claims, is exactly the structure of time dilation — the effect in special relativity where a clock on a fast-moving spaceship ticks slower than one on Earth. And from similar tricks involving something called commute time (roughly: the expected number of steps to go from state A to state B and back), you can cook up distances that behave like Euclidean geometry. So the whole picture of relativistic space-time might just be a statistical shadow of observers watching each other’s experience-chains tick by.
He’s not claiming this is done. He’s claiming he can see it from here.
The bridge to quantum theory
There’s a natural objection: quantum theory uses unitary matrices, which are a very specific kind. Markov chains are generally not unitary. So how do you get quantum physics from this?
Hoffman’s answer: some Markov chains happen to be unitary, and more importantly, when you look at the long-run asymptotic behavior of Markov chains, you find that the mathematical structures describing them become identical in form to the wave functions of quantum particles. He and Prakash showed this back in 2014. The idea is that quantum theory is the long-run limit of a Markov-chain dynamics, and Markov theory is the more detailed moment-to-moment story. Quantum theory gives you statistics after the chain has settled. Markov chains give you the frame-by-frame.
And what about the no-cloning theorem — the famous quantum rule that says you can’t perfectly copy an unknown quantum state? Hoffman says if you read the proof carefully, it doesn’t actually need quantum theory. It only needs linearity. Markov chains are linear. They have their own no-cloning theorem. So that’s one fewer thing quantum physics has that Markov chains lack.
Chris Fields raises a warning flag
Fields — who seems to be the designated skeptic in this conversation — points out that there are situations in quantum theory called contextuality, where joint probabilities simply can’t be defined in the usual way, and he’s not sure Hoffman’s Markov-chain machinery can handle that. Hoffman doesn’t have a complete answer but suggests that if the trace logic has pairs of matrices that don’t commute (meaning: doing A-then-B gives a different result than B-then-A), that might be the handle.
Fields also calls out that when the positive-geometries physicists say “space-time is doomed,” they mean something narrower than Hoffman is making it sound — they’re talking about space-time as a physical stage with local interactions, not about unitarity as a mathematical principle. Hoffman takes the correction. This is the healthy kind of scientific argument: everyone on the call can hear when they’re overreaching.
Levin’s hidden platonic realm
Michael Levin is mostly listening, but when he speaks, he brings his own weirdness. His lab studies planaria — flatworms you can cut in half and each half regenerates into a full worm. And if you zap the worms with the right electric fields, you can make them regenerate with two heads. How does the worm “know” what to build? Where is that blueprint stored?
Levin has been developing an idea that biological patterns live in a kind of Platonic space — a realm of mathematical patterns that physical systems somehow reach into and pull down. It sounds mystical, but he’s very insistent it’s a prediction-generating scientific framework. His lab can measure cases where organisms seem to get “more than they paid for” — more memory, more compute, more problem-solving — given conventional accounting.
And Hoffman lights up at this, because the trace logic gives him a concrete proposal for what that Platonic space might be. The hidden states — the B, C, D submatrices that the trace observer cannot see — are the Platonic space. They’re doing invisible work that shows up in the visible trace as seemingly unmotivated intelligence. The worm regenerating correctly is the trace. The actual computation is happening in states the biologist can’t directly observe.
“If somehow all we’re seeing is the planaria in our trace, we’re not seeing beyond what we can see. There’s a whole Markov realm of intelligence out there that is projecting down into what we can see, which is just a planarian.”
Levin is delighted. He wants to test the idea by taking simple random Markov chains — ones nobody designed for any purpose — and running the standard behaviorist test battery on them. Habituation, associative conditioning, delayed gratification, illusion, counterfactual reasoning. His bet is that these capacities will show up in traces of systems that nobody trained, evolved, or programmed. If they do, it’s evidence that intelligence is the default behavior of dynamical systems, not a rare achievement.
Where they end up
The meeting ends like good collaborations do: nothing proven, a lot to do, everyone agrees to pick a simple test case and meet again. Robert Prentner notes that he’s folding this stuff into a forthcoming paper on the free energy principle. Hoffman promises to send preprints. There’s a conference in Spain in July.
The interesting thing about watching four smart people talk for an hour and a half is that none of them is claiming to have the answer. They’re claiming to see a direction — consciousness-first, observer-first, with math that might actually be tractable — that hasn’t been taken seriously in mainstream physics or consciousness research. Whether it pans out is an empirical and mathematical question. But it’s not hand-waving. There’s an equation, a logic, and a program of work.
Claude’s Take
The honest assessment is that this video is a research seminar in the middle of someone’s work, not a finished theory, and the fermentation of it depends heavily on where you stand on Hoffman’s larger worldview.
The math is real. Markov chains, trace operations, and their partial orderings are legitimate mathematics that has been studied for decades. Hoffman and Prakash’s specific claim — that the trace relationship defines a partial order on Markov chains, and therefore a logic — is the kind of observation that’s either provably correct or provably wrong, and Hoffman says it’s on the “correct” side. I have no reason to doubt that, and the connection to Bayesian inference via the “meet” operation in the logic is the kind of thing that makes mathematicians sit up. The link between lambda two (second-largest eigenvalue) and search efficiency is standard spectral graph theory, not news, but embedding it in this framework is novel.
The “get space-time and quantum theory from this” claim is where it gets speculative fast. Hoffman is honest that it isn’t done. He has pieces: the counter trick gives something that looks like time dilation, commute times give something that looks like distance, and he and Prakash showed in 2014 that asymptotic Markov eigen functions resemble free-particle wave functions. But “resembles” and “is” are very different. Chris Fields, who is himself a serious physicist, repeatedly and politely flags the gap between “we can reproduce some observables” and “we’ve derived the theory.” Hoffman doesn’t quite answer those objections — he accepts them and moves on. I’d trust him more than most philosophers attempting this leap, because he knows the math, but I wouldn’t bet on the relativity derivation landing as cleanly as he hopes.
The consciousness-first framing is the part where you have to decide whether you’re signing up for a philosophical package. Hoffman is a principled and careful thinker about perception, and his “interface theory” work is genuinely worth taking seriously — his core point that evolution optimizes for fitness, not accuracy, and therefore that perception is a user interface rather than a literal window, is well-defended and has empirical support. But the jump from “perception is an interface” to “space-time is nothing but an interface built by consciousness” is a much bigger step. It’s metaphysically ambitious in a way that some people will find thrilling and others will find suspiciously convenient. When he says “Hoffman is just an avatar in a space-time headset created by consciousness,” that’s not a scientific claim yet — it’s a research hypothesis with heavy philosophical priors baked in.
The Markov blanket upgrade to “trace blanket” is interesting but underdeveloped. He openly says he doesn’t know how to construct the self cleanly in this framework and is hand-waving toward pleasure-pain gradients and learned regularities. Compared to the existing Friston framework — which is itself criticized for doing a lot of hand-waving — this isn’t obviously better yet, but the recursive structure might give it legs.
What’s most genuinely novel is the recursive move. The observation that the space of policies is itself a Markov chain with its own trace logic, and therefore that meta-policies, meta-meta-policies, etc., form an infinite tower — that’s a nice conceptual move, and it matches how people actually seem to reason (I’m not just choosing what to do, I’m choosing how I’m choosing). Whether it survives translation into a formal model that predicts something concrete is open.
Michael Levin’s side is the most empirically grounded and also the strangest. His planaria work is real and widely respected, and his Platonic-space framing, while provocative, is trying to explain an actual gap in conventional biology — things like bioelectric pattern memory that don’t fit neatly into DNA-centric accounts. Whether “Platonic space” is the right metaphor or just a placeholder for “stuff we don’t understand yet” is up for debate. His openness to testing these ideas on simple random systems is the scientifically honest part: if random Markov chains show habituation and conditioning without training, that’s a genuine discovery regardless of interpretation.
Two specific things to watch. First, Hoffman claims Bayes’ rule falls out of the “meet” operation in the trace logic. If that’s true in the clean way he implies, it’s a lovely result — Bayesian inference is a cornerstone of modern statistics and machine learning, and deriving it from a more primitive principle would be a big deal. Second, the claim that the set of all Markov chains forms a “Markov polytope” that is itself a positive geometry related to the amplitudehedra that physicists are studying — that’s testable mathematics. Either it’s true or it isn’t.
Overall: I’d call this a “watch this space” situation. The mathematical core is respectable, the broader program is ambitious and partly speculative, and the cross-pollination with Levin’s biology is the most scientifically interesting part because it suggests experiments. Anyone who tells you Hoffman has proven that space-time is built by consciousness is overselling it. Anyone who tells you it’s obvious nonsense is underselling it. The correct stance right now is curiosity and a demand for the preprints.
One last observation: the vibe of this meeting is worth noticing. Four serious researchers, none of them needing to impress the others, openly pointing out gaps in each other’s arguments and building together. This is what pre-paradigmatic science looks like when it’s being done well — not slick, not finished, but not crank either. Whether it leads anywhere is a question for the next five years.