Why Physics Without Philosophy Is Deeply Broken... | Jacob Barandes

ELI5 / TLDR

Jacob Barandes is a Harvard physicist (and co-director of the graduate program there) who thinks the standard textbook recipe for quantum mechanics — wave functions, Hilbert space, superposition, collapse — is not just weird, it’s quietly broken. He spent years building an alternative called “indivisible stochastic processes,” which is a long way of saying: things have actual configurations at all times, the laws are just probabilities about how those configurations change, and the whole Hilbert-space machinery falls out as a clever rewriting of those probabilities, not the other way around. No wave function. No collapse. No particle being in two places at once. The interference patterns and the spookiness all turn out to be artifacts of trying to squeeze a fundamentally non-Markovian story (one whose laws don’t only depend on the present) into a Markovian formalism (one that pretends they do). It is a real, technical, peer-reviewed program — not a crank’s pamphlet — and it sits inside a larger argument that physics as a discipline lost something important when it stopped doing philosophy.

The Full Story

What “physics without philosophy” actually means

Barandes splits his work in two directions. One is physical philosophy — using results from physics to settle ancient questions in metaphysics. (Example: special relativity strongly implies the future already exists, because two observers moving at different speeds will disagree on which events are happening “now” — so if you stitch all their nows together, you get the whole space-time loaf. The 1960s philosopher Hilary Putnam made this argument; it’s hard to refute.) The other direction is philosophical physics — using the tools of philosophy to do better physics.

What are those tools? Sharpening definitions until they squeak. Looking for the assumptions you didn’t notice you were making. Subjecting every step of your reasoning to what Barandes calls “rigorous scrutiny.” Thought experiments. Counter-examples.

The greatest philosopher of physics of the 20th century was Albert Einstein.

That’s a quote from a philosopher of science Barandes likes. The point is that Einstein’s biggest moves came from staring at a definition until it cracked. What exactly is an inertial reference frame? What exactly does it mean for two events to happen at the same time? He’d read all three of Kant’s Critiques by sixteen. Schrödinger argued with Heisenberg about who was reading Kant correctly. Bohr was a working philosopher. Heisenberg’s last book is full of neo-Kantian metaphysics. The people who built quantum theory and relativity were thoroughly soaked in philosophy. Modern physicists like Neil deGrasse Tyson now ask “what has philosophy done for physics in 30 years?” and Barandes politely lists: the no-cloning theorem (1982, partly from a philosopher), Bell’s theorem and everything downstream of it, decoherence (introduced by David Bohm in 1951 in the most philosophical chapter of his quantum textbook), the no-signaling theorem, the entire foundations-of-quantum-mechanics field that quantum information now leans on. Philosophy keeps writing the cheques and physics keeps cashing them without saying thank you.

That’s the framing. Now to the actual physics.

What’s broken in textbook quantum mechanics

The standard recipe, the one in Griffiths or Sakurai, has roughly five axioms. Don’t worry about all of them — the shape is what matters.

Every quantum system has a state, written as a vector in something called Hilbert space. Think of it like a glorified arrow living in a very high-dimensional space, where the components of the arrow can be complex numbers.
The state evolves smoothly in time according to the Schrödinger equation. Smoothly meaning: if you know the state right now, the equation tells you the state a moment later. (This is called Markovian evolution — the future only depends on the present, not the past.)
The things you can measure (position, momentum, spin) are represented by mathematical objects called operators.
When you measure, you get one of a discrete menu of possible outcomes, with probabilities given by the Born rule.
After the measurement, the state suddenly collapses to match the outcome you got.

For tabletop experiments — atoms, lasers, particle accelerators — this works astonishingly well. So what’s the problem?

The problem is that the recipe contradicts itself once you try to apply it to anything bigger than a microscope slide. The clearest version is Wigner’s friend. Picture a sealed box. Inside the box is a friend who measures some quantum system. According to axiom 5, the friend causes a collapse. But from outside the box, you haven’t measured anything, so according to axioms 1–2, the entire contents of the box (friend included) should still be in superposition. Did collapse happen or not? The axioms simply do not answer. They’re ambiguous, and the ambiguity isn’t a matter of interpretation — it’s a logical hole in the middle of the theory.

There’s a difference between a theory being unintuitive or exotic or eccentric and a theory being inconsistent.

This is the move worth pausing on. Newtonian mechanics is unintuitive (objects in motion stay in motion forever? gyroscopes? centrifugal force is fake?). General relativity is wildly unintuitive. We’re fine with unintuitive. Eccentric friends are great. What we’re not fine with is a friend who, when asked a simple question, contradicts themselves. That’s quantum mechanics outside the microscopic regime: not strange, broken.

There’s a second problem Barandes calls the category problem. The textbook axioms only predict one kind of thing: the readings on measuring devices. Probabilities of dial positions. But the universe is full of stuff that nobody is measuring — primordial gas mixing, birds foraging, you falling in love. Does the theory describe those? Strictly speaking, no. The axioms are silent. Either we accept that quantum mechanics only covers a thin slice of reality (sad), or we have to argue that all of those things secretly are measurements (uphill), or we need a different theory. Decoherence is sometimes wheeled out to plug this gap, but decoherence by itself doesn’t single out a single outcome — it just makes the math look like outcomes have happened. Something is still missing.

The two prejudices to drop

Barandes thinks two assumptions have been silently doing damage for ninety years.

Prejudice one: laws should be Markovian. That is, the present state alone should determine the future. This feels obvious because it’s how every differential equation in physics class works. But it’s not actually mandatory. Even Newtonian mechanics is secretly non-Markovian: to predict where a thrown ball goes, you need its position and its position an instant ago. We hide that by inventing a quantity called “velocity” — really just (position now − position a moment ago) divided by a tiny time — and bolting it onto the state. Same information, dressed up. So Newton-style physics fakes Markovianity by doubling the bookkeeping.

Prejudice two: observables are all-or-nothing. Either everything you can measure has a value sitting there waiting to be discovered (in which case ordinary probability theory works fine), or nothing does and you have to give up classical probability entirely. There’s a famous result called the Kochen-Specker theorem that seems to force the second option for quantum systems. Barandes illustrates it with a tic-tac-toe game where the rules are paradoxical — the rows always have an even number of X’s, the first two columns also even, but the third column always odd, which is mathematically impossible if there’s a fixed board hiding in someone’s head. Quantum systems pull off this trick. The standard reading is: there’s no fixed board. Observables don’t have values until measured.

Barandes pushes back. Maybe the right reading is that some observables have values waiting to be revealed, and others are emergent patterns that only appear when a measuring device couples to the system. John Bell coined a word for the first kind: beables (things that be). Barandes adds a word for the second kind: emergibles. From the outside, the dial on the measuring device doesn’t know which is which. Both produce honest readings with honest probabilities. But under the hood, only the beables were really there.

This is exactly how Bohmian mechanics already handles it: in Bohmian mechanics, particle positions are beables (they really exist), but momentum is an emergible (the number you read off depends on the apparatus, not just the particle).

The one-sentence theory

Every system has an actual configuration belonging to some menu of possible configurations called the configuration space. And the dynamics, the laws by which the configuration changes with time, is characterized by a sparse set of directed conditional probabilities that generically fail to be divisible in time.

That’s the whole picture. Two pieces: configurations (where the system actually is) and conditional probabilities (the chance, given that it’s here now, of being there later).

The strange word is indivisible. In ordinary probability — say, the weather — if you want to know the chance of rain on Friday given sun on Monday, you can break the calculation into chunks: chance of Tuesday’s weather given Monday’s, then Wednesday given Tuesday, and so on, multiplying through. The probabilities chain together cleanly. That’s divisibility. Markovian processes are divisible by definition.

In Barandes’s framework, the laws don’t chain like that. The probability of being at point B at time 2 given you were at point A at time 0 is something you’re given as a primitive — and you can’t reconstruct it by splicing together “A to somewhere at time 1” and “somewhere to B from time 1 to 2.” Try and you get the wrong answer. The dynamics is genuinely non-Markovian, and not in the polite way where you can fix it by enlarging the state space. Truly, irreducibly indivisible.

Where the wave function comes from

Here’s the cleanest payoff. Barandes proves a mathematical theorem (the stochastic-quantum correspondence, in his 2023 arXiv paper of that name) that any indivisible stochastic process can be rewritten as a system in Hilbert space. State vectors, density operators, Born rule, unitary evolution, the works. They are not different theories — one is just a mathematical re-expression of the other.

Why would you do that re-expression? For the same reason Newton’s mechanics gets rewritten as Hamiltonian mechanics, even though they describe the same physics. Hamiltonian mechanics looks weirder — coordinates and momenta on equal footing, complex numbers sneaking in, freedom to rotate variables in mind-bending ways — but it makes certain calculations enormously easier. The Hilbert-space picture of quantum mechanics is the same kind of upgrade, but applied to a probabilistic theory rather than a deterministic one.

And the analogy is structural, not poetic. Hamiltonian mechanics introduces enhanced symmetries (canonical transformations) that mix what you mean by position and momentum; Hilbert-space QM has corresponding symmetries (basis changes) that mix what you mean by different observables. Both formalisms develop complex numbers organically. Both are doing the same trick: taking a non-Markovian theory and faking Markovian evolution by inflating the state.

Once you accept that, a stack of mysteries dissolves:

Why is the Schrödinger equation linear? Because ordinary probability theory is linear. The law of total probability — combine initial probabilities with transition probabilities to get final probabilities — is a linear operation. Linearity in Hilbert space is just classical probability arithmetic in fancy clothes.
Why is the evolution unitary? Because of a theorem (Stinespring’s dilation theorem, from the 1950s) that says you can always represent a generic stochastic process as unitary evolution if you slightly enlarge the Hilbert space.
What are interference, superposition, and the off-diagonal terms in density matrices? Bookkeeping debt. They’re the price you pay for forcing a non-Markovian story into a Markovian formalism. They’re not “really there” in the world.
What is collapse? Nothing. There was never a wave function. The particle had one definite location all along. What looked like collapse is just classical marginalization — you had a joint probability distribution over the system and a detector, you sum out the detector you can no longer access, and what remains looks like a process that’s suddenly divisible at that moment. On the Hilbert-space side, that’s decoherence and projection. On the stochastic side, it’s just integrating out a variable. No magic.

The double slit, demystified

Set up a coarse-grained double slit: a particle is either in the upper half or lower half of a chamber, hits a wall with two holes, lands in the upper or lower half of a screen. Fire many particles, build a histogram, and you get the famous interference pattern — bands of dark and light, as if a wave had passed through.

In the indivisible picture, the particle has a definite position the whole time. It goes through one hole, not both. There is no wave. The interference pattern emerges because the law that describes the joint motion from start to end is not divisible at the holes — you can’t say “given that it’s at the upper hole now, here’s its onward law.” That law isn’t supplied. The dynamics is one continuous indivisible step from emitter to screen, and a particular family of such dynamics produces, statistically, the interference bands.

Now add a which-path detector — a small system that flips state if the particle takes the lower hole, leaves itself alone if upper. If the detector is reliable, the joint dynamics suddenly becomes divisible at the holes. The interference vanishes. The particle still has a definite position, still goes through one hole. Nothing collapsed. The detector simply made the indivisibility look divisible by storing which-path information classically.

There was no wave function, there was never a superposition, there was never a need to get anything to collapse.

That’s the clean line. The whole spooky cathedral of Copenhagen was, on this view, the wrong picture all along.

Where mainstream physics pushes back

The mainstream reaction is not “this is crank stuff.” Barandes is a tenured Harvard physicist publishing in respectable venues (the stochastic-quantum correspondence is at arXiv:2302.10778). Sean Carroll has had him on his podcast and treats the work seriously. The pushback is mostly about whether the program survives contact with the harder problems:

Quantum field theory. Standard QM is non-relativistic and finite-dimensional. Barandes’s correspondence is proven there. Extending it to QFT — fields, infinite degrees of freedom, particle creation and annihilation — is open work. He acknowledges this is the next mountain.
Bell’s theorem. Bell proved that no local theory with hidden variables can reproduce quantum predictions. Barandes’s view has beables (variables that exist before measurement), so it lives in the same space Bell was constraining. The escape is that the non-Markovian dynamics let it dodge the conditions of the theorem in subtle ways. This is the kind of claim that needs careful technical defence; Curt and Barandes promise a follow-up conversation on it.
Many-worlds advocates (Sean Carroll, David Deutsch) argue that just taking the Schrödinger equation seriously and dropping collapse is simpler than introducing a parallel stochastic-and-Hilbert duality. Barandes’s reply is that a beable-based theory is closer to how the world actually feels (one outcome, not infinitely many branching ones) and that the indivisible framework explains why the wave function looks the way it does, instead of taking it as primitive.
Bohmian mechanics advocates (Tim Maudlin, Sheldon Goldstein) might say: we already had a beable-based theory, why this one? The answer is that Bohmian mechanics requires a separate guiding wave on top of the particles, and gets awkward when you try to make it relativistic; Barandes’s framework needs no guiding wave, just the configuration and the conditional probabilities.

The honest summary: it’s a serious, contested program. Some philosophers of physics (Maudlin, Tim Norsen) take it seriously. Most working physicists haven’t engaged with it because they don’t engage with foundations work at all. Whether it survives extension to QFT will determine if it’s a footnote or a fork in the road.

Key Takeaways

The Dirac–von Neumann axioms (Hilbert space + unitary evolution + Born rule + collapse) are inconsistent in regimes like Wigner’s friend, not just unintuitive. There’s a real logical hole, not a philosophical squabble.
Two unexamined prejudices have shaped quantum mechanics for ninety years: that fundamental laws must be Markovian, and that observables are either all pre-existing or none are. Both can be dropped.
An indivisible stochastic process is a probabilistic system whose conditional probabilities don’t chain together — you can’t slice the dynamics into Markovian steps. This is genuinely different from “non-Markovian with extra memory.”
Every indivisible stochastic process has a Hilbert-space representation (the stochastic-quantum correspondence). The wave function, superposition, interference, and collapse are all artifacts of forcing a non-Markovian theory into Markovian form. They are bookkeeping, not ontology.
Beables are observables whose values exist independently of measurement (like position in Bohmian mechanics). Emergibles are observables whose values only emerge from the joint dynamics of system and apparatus. The dial doesn’t know the difference; the metaphysics does.
The double slit’s interference pattern is explained without waves. It comes from the law’s failure to be divisible at the slits. Adding a which-path detector restores divisibility, kills the interference — no collapse required.
Decoherence on the Hilbert-space side corresponds to classical marginalisation on the stochastic side. Same operation, different language.
Einstein, Bohr, Heisenberg, Schrödinger were all fluent in philosophy. Modern physics’ allergy to philosophy is recent and probably hurting it. The areas of physics where progress has stalled (foundations, quantum gravity) are the exact areas where rigorous philosophical scrutiny is most needed and least funded.

Claude’s Take

Score: 9/10. This is a rare thing — a long-form interview where a working physicist patiently dismantles the standard story without crankery, without bombast, and without leaving you in the cold. Barandes is unusually clear. He builds the stack from “what’s an observable” to “here’s why the Schrödinger equation is linear” without skipping rungs, and he’s honest about what’s open (quantum field theory, gravity, the trickier corners of Bell). The fact that the conversation is over seven hours total is itself the point: foundations work cannot be done in a five-minute soundbite, and one of the implicit theses of the episode is that podcasts of this length are now where serious physics-and-philosophy conversation actually happens.

What to take with caution. Barandes’s program is genuinely contested. The mathematics — the stochastic-quantum correspondence — is well-defined, peer-reviewed, and unsurprising once you see how it works (a structural analogy with Hamiltonian mechanics that physicists have noticed in pieces for decades). The interpretive claim — that this is the right way to read what quantum mechanics is telling us — is much more contested. Most working physicists would shrug and say “we don’t care, the predictions are the same.” Most foundations people are split between Bohmians, Everettians, QBists, and a long tail. Barandes is a new entrant and his framework will be judged over the next decade by how well it handles QFT and gravity. So treat it as a live, serious, unsettled program — not as the resolution of foundations.

The case against philosophy in physics is also weaker than its loudest exponents (Tyson, Krauss, Hawking-late-period) make it sound. Barandes’s list of philosophy’s contributions is a fair representation of the historical record. The real question isn’t “does philosophy contribute?” but “what’s the right ratio of speculative philosophical scrutiny to empirical work, given a particular subfield’s data situation?” In foundations of QM, where new experimental data is rare and theoretical assumptions are everything, the ratio should tilt heavily toward philosophy. In condensed matter, less so. That’s the more honest version of his argument.

One small caveat. The transcript has a few stretches — especially toward the end — where Barandes drifts into general advice for aspiring physicists (read the footnotes! be skeptical of textbook authors!) that’s good but not load-bearing for the main argument. The first half is the meat.

If you’re going to learn one thing from this: physics has a habit of presenting its own interpretation as if it were data. The line where calculation ends and metaphysics begins is rarely marked. The discipline most trained to spot that boundary is philosophy.