How Intelligence Arises in People and Machines | The Caravan Conversations Ep1

ELI5/TLDR

A neuroscientist sits down with an old friend to argue, slowly and carefully, that the mind is not a thing but a process. Thoughts are just patterns of electricity in connected cells. Consciousness is not the driver, it is the surface foam that sometimes appears when those patterns get loud enough. The same trick, scaled up in silicon, is what makes large language models work. There is no system two, no inner manager, no soul. Just one system, deep enough to contain everything we mean by “us”.

The Full Story

The conversation is between Hartosh Singh Bal and Gautam Suri, a computational neuroscientist at San Francisco State and Stanford, and co-author with Jay McClelland of The Emergent Mind. They were friends for forty-five years before either of them wrote a book. Suri does most of the talking. Bal pushes the gentle, well-timed objections.

Wetness, ants, and what “emergent” actually means

Emergent is one of those words that gets thrown around until it stops meaning anything. Suri pins it down: a property of the whole that is not present in the parts. Water is wet because hydrogen bonds let sheets of molecules slide past each other. One molecule of water is not wet. Six are not wet. Get enough together and wetness shows up.

The ants are the better example. Set up an obstacle on the trail between an ant nest and a food source, with a long way around and a short way. In fifteen minutes, ninety percent of the ants are taking the short route. Nobody told them. No queen issued orders. They each just lay a chemical signal — a pheromone — and follow the strongest trail they smell. The shortest path emerges from a population of ants that have no concept of paths.

“The ants don’t have a conception of the shortest path, but the ant colony is able to find the shortest path around obstacles.”

Hold that thought. The book is making the same move at a larger scale. The mind has properties no single neuron has.

Mind is not consciousness

This is the first thing to surrender. Most people, when they say “mind”, mean the running narration in their head — the bit that decides what to wear, who to marry, which Uber to book. Suri’s claim is that this narration is largely a story we tell ourselves after the fact.

He runs through the standard catalogue of evidence. The thirst experiment: when you get up for a glass of water, what actually moves you is a population of neurons in the hypothalamus detecting high blood salinity. The “I’m thirsty” thought is downstream, often arrives late, sometimes does not arrive at all.

Then there is the 1977 paper by Nisbett and Wilson, Telling More Than We Know. Four identical stockings are arranged left to right. People pick the rightmost one half the time. When asked why, they explain — the colour, the texture, the softness. The stockings are identical. The explanations are confabulation.

The split-brain experiments make it sharper. Cut the corpus callosum, the bundle of nerves between the two hemispheres, and you can show a message to one half of the brain that the other half cannot see. Flash “go to the bathroom” to the right hemisphere. The patient gets up and walks to the bathroom. Ask why. They do not say “I don’t know”. They say “I think my hands are sticky”. The action has a cause. The conscious explanation is a fiction the left brain spins up in real time to make the action feel chosen.

“We have people who are trying to commit suicide with one hand and prevent the suicide with the other hand.”

So when Suri uses the word “mind”, he means something narrower and stranger: patterns of activation in the brain that lead to action. Consciousness optional.

Intelligence is flexibility

Intelligence is then defined cleanly: flexibly responding to context. An octopus, with its eight arm-brains plus a central brain, is intelligent because it adapts. A spider weaving its web is intelligent in a narrow sense — informed by temperature and pressure — but cannot regulate the weaving once it starts.

By this definition, large language models are intelligent. Two years ago, ask one whether you can withdraw money from a bank you approached with a fishing pole, and it would fumble. Now it tells you, no, that is a river bank. The model can hold the birthday problem in its head and answer it correctly. It flexibly responds to context. It is intelligent. It is, Suri insists, almost certainly not conscious. Those are different claims.

Why the symbolic AI dream collapsed

For seventy years we tried to build intelligent machines out of if-then-else statements. Take a fact, apply a rule, produce a conclusion. The dream came from mathematics — Socrates is a man, all men are mortal, therefore Socrates is mortal. If you can capture human reasoning as symbolic logic, you can simulate it.

It did not work. Billions of dollars vanished into what is now called the AI winter. The deeper problem, which Suri flags with a lovely observation: humans are faster at “all men are mortal, your grandfather is a man, therefore your grandfather is mortal” than they are at the same syllogism with made-up words. The logic is identical. The performance is not. Human cognition is not running on symbolic rules. Something else is going on underneath.

Neurons, activation, and the trick of patterns

The something else is connectionism. A neuron either fires fast or slow. That rate is its activation — one number. Neurons connect to other neurons. When green-related neurons fire, they pull tree-related neurons along with them. Strip away the biology and you have units that hold a number, and connections that carry influence. That is a neural network.

The clever bit is how ideas get represented. Three neurons with on-off activations can hold eight possible patterns. Add more neurons, allow activations to take values between, say, -0.2 and 1, and you have a vast space of possible patterns. A particular pattern stands for “sofa”. A slightly different pattern stands for “couch”. Crucially, similar things in the world have similar patterns. The brain figures this out on its own, and we do not really know how.

“A particular pattern represents a particular idea, and a different pattern represents a different idea… similar things in the world can have similar patterns.”

Where do these patterns come from? Suri walks through a baby. Light hits the eyes, smell hits the nose, milk hits the mouth. Each sense organ has innate wiring to particular brain regions. When the mother arrives, all these neurons fire together. Co-firing strengthens connections. After enough exposures, the baby has a stable pattern for “mother”. One day the father picks her up — different smell, different face — and the pattern does not match. Yuck.

Bias works the same way. The machinery that lets you form the concept “tree” is the machinery that lets you decide, on sparse data, that a particular group is dangerous. The difference is that bias gets supercharged by ancient circuits for group identity and tribalness. We would be cognitively lost without the concept-forming machinery. The trouble is it forms concepts fast, on too little evidence, and emotion amplifies them.

Where control comes from

If everything is activation flowing through connections, what stops you from raising your hand the moment someone says “raise your hand”? The game of Simon Says is the test case. The answer, Suri offers, is not a separate executive system. It is more activation. A goal — say, “follow the Simon Says rule” — is itself a pattern in the frontal cortex, and it actively tamps down the action pattern when conditions are not met.

The cortex is the outer covering of the brain, about the size of a newspaper when unfolded, crammed into the skull by being folded. It can hold many patterns simultaneously. Bottom-up signals from the senses activate goals. Goals fold back down onto actions. No manager required. Just activations interacting with activations, boosted by neuromodulators.

Neuromodulators, and what dopamine is not

Neurotransmitters carry signals between two neurons. Neuromodulators flood the whole network and change how it behaves. Dopamine, serotonin, norepinephrine — these are the chemicals that take the brain out of business as usual.

Suri is firm about a common confusion: dopamine is not the reward chemical. Reward is the post-meal heaviness, the sleep after a good meal. Dopamine is the wanting. It strengthens the activations and connections that drive you toward a goal. Serotonin, roughly speaking, is the opposite — it supports cessation, calmness. When SSRIs flood the brain with serotonin, motivation often dampens. The two systems are coupled.

Why neuromodulators matter for the larger argument: they are how the brain does one-shot learning. See a tiger at the watering hole once, learn forever. You do not need to feel fear for this to work. The withdrawal circuits fire, the neuromodulators amplify them, the pattern locks in. The conscious feeling of fear rides along, but it is not the cause. There is even a clinical case of a patient who had involuntary orgasms and experienced them as painful, not pleasant. The bodily process was identical. The subjective tag was opposite.

No system two

Daniel Kahneman split cognition into two: System 1 (fast, associative) and System 2 (slow, deliberative). Suri’s claim is bigger and stranger. There is no System 2. There is only System 1, with System 2-like behaviour emerging from the interactions inside it. Large language models, as far as anyone can tell, are pure System 1 — pattern association — and they do mathematics now. Poetry and love and painting, Suri thinks, come from the same dry plumbing.

“Are you sure that poetry and love and painting and mathematics come from these rather dry activations flowing through connections? I believe so.”

Descartes was wrong in a useful way

Descartes once walked through the gardens at Versailles, stepped on a stone, and watched a statue’s arm move. The gardener explained — pressure, tubes, hydraulics. Descartes built his theory of mind on this. A tube runs from your finger to your brain. Heat compresses the fluid. The brain gives the signal to pull back.

Wrong in every detail, but right in the move. Descartes was the first to try a mechanistic answer for the body, instead of resorting to spirit. Then he flinched. For the higher functions — love, painting — he kept the magic. That dualism is still the default human position. Dogs are machines. We are touched.

Suri thinks we can drop the second half. We are processes. Like photosynthesis, like water running downhill, like the trees, like the stars. Amazing processes of enormous complexity, but processes.

Ethics without souls

If we are deterministic processes, what holds a society together? Suri’s answer is utilitarian and unsentimental. Good people and bad people are not real categories. People are connections — the ones they were born with and the ones their experiences carved. So why punish? Not for moral reasons. For practical ones. Cultures need checks and balances. We agree as a society on the behaviours we want to encourage and the ones we want to discourage, and we build mechanisms — including punishment — to bias the system toward the former.

He extends it to himself. If he does not want to eat chocolate, he does not buy it in the store, because he knows that in a moment of hunger his activations will win. Checks and balances on himself. Checks and balances on each other. Checks and balances, he adds quietly, on the machines we are now building.

The hard problem

Bal saves the biggest objection for the end. None of this explains consciousness. How does any activation pattern, however clever, give rise to the smell of rain after a dry spell, which has a name — petrichor — and a felt quality nothing in the equations predicts?

Suri does not pretend to solve it. He points out that physics has been going for centuries and we still cannot say what mass is, or what charge is. We just have predictive theories that work. The same posture is available for consciousness. He can map which patterns tend to surface into awareness (the strong, stable, long-lasting ones, supported both top-down and bottom-up). He can speculate about its function. He can do this without cracking the hard problem.

“I don’t know what consciousness is, but does that mean I shouldn’t study the mind? Not at all.”

He does not think it is unsolvable. He just does not know where to start.

Key Takeaways

Emergence definition: a property of the whole not present in the parts. Wetness from hydrogen bonds, shortest-path-finding from pheromone-laying ants
Mind vs consciousness: Suri defines mind as patterns of activation that lead to action. Consciousness is optional and often arrives after the action is already in motion
Confabulation evidence: Nisbett and Wilson’s 1977 stocking experiment (identical stockings, invented reasons); split-brain patients who instantly fabricate explanations for actions caused by signals their left hemisphere never saw
Intelligence definition: flexibly responding to context. By this definition LLMs are intelligent (the “fishing pole at the bank” example shows context-sensitivity that did not exist two years ago)
Why symbolic AI failed: humans reason faster about familiar entities than identical syllogisms with made-up words — logic is not the underlying substrate. The AI winter followed seventy years of if-then-else dead ends
Activation patterns: ideas are patterns across populations of neurons. Similar things in the world get similar patterns. The brain does this similarity-mapping spontaneously and we do not understand how
Concept formation: co-firing strengthens connections (Hebbian-style). Bias uses the same machinery as concept formation, supercharged by tribalness circuits
No System 2: pace Kahneman, Suri argues deliberation emerges from System 1 association. LLMs are pure association and now solve previously unsolved math problems
Dopamine is wanting, not reward: amplifies pursuit-circuit activation. Serotonin supports cessation. SSRIs flatten motivation as a side effect
One-shot learning via neuromodulators is how brains learn from a single tiger at the watering hole — without conscious fear being the cause
The patient with involuntary, painful orgasms: the bodily process and the subjective tag (pleasure vs pain) can dissociate, suggesting felt qualities ride on top of mechanism rather than driving it
The hard problem of consciousness is real but not a prerequisite for studying the mind, in the same way we do physics without knowing what mass actually is

Claude’s Take

Suri is unusually disciplined for this genre. The “consciousness is overrated” position has been bloated for years by people who confuse a hot take with an argument. He grounds every claim in specific experiments — Nisbett and Wilson, the split-brain work, the involuntary orgasm case, the SSRI mechanism — and he distinguishes carefully between what is established (mind precedes consciousness) and what is speculative (no System 2 at all).

Where he oversells: the “LLMs are intelligent by my definition, just not conscious” line is true but does a lot of rhetorical work. Calling current models intelligent because they handle context-sensitive prompts skips over the harder question of whether their flexibility is the same kind that emerges in biological systems, or a sophisticated interpolation that mimics it from the outside. He waves at this but does not press. The promise of a Part 2 on LLMs presumably gets there.

The “no System 2” claim is the boldest move in the conversation and gets the lightest defence. Saying “LLMs are System 1 and they do math now, so deliberation must emerge from association” is suggestive, not decisive. Chain-of-thought reasoning in LLMs is itself a kind of scaffolded deliberation, and dismissing the felt difference between fast intuition and slow reasoning as just-more-activation is the kind of move that needs a thicker argument than the one offered here.

The ethics-without-souls section is where the framework earns its keep. The shift from “punish bad people” to “design cultures with checks and balances” is the practical pay-off of taking determinism seriously, and Suri delivers it without sliding into either moralism or nihilism.

Score: 8/10. Patient, evidence-anchored, free of the usual breathlessness. Loses a point for the LLM-comparison hand-waving, which the format invites but the speaker does not push back on. The Caravan made a good call letting two old friends talk slowly for seventy-seven minutes.