How Intelligence Arises In People And Machines Caravan Conversations Ep1

TITLE: How Intelligence Arises in People and Machines | The Caravan Conversations Ep1 CHANNEL: The Caravan Magazine DATE: 2026-05-18 ---TRANSCRIPT--- In the interest of full disclosure, today I’ll be talking to an old friend, a friend of maybe more than 40, 45 years. Mhm. He’s also a distinguished neuroscientist. He is a professor at San Francisco State University, where he has his own lab. And he’s a distinguished scientist at Stanford University. He’s also the author of this book, The Emergent Mind, How Intelligence Arises in People and Machines. And I think a look at the cover should give you some idea of what the book is. The names are a who’s who of uh sort of neuroscience. His co-author, Jay McClelland, is one of the foremost neuroscientists in the world. I can now claim, having co-authored a book with Gautam Suri, that Jay McClelland and Hartosh Singh Bal are co-authors of Gautam Suri. Gautam, welcome. Thank you. Uh The Emergent Mind, How Intelligence Arises in People and Machines. Mhm. Can we just start here? Why The Emergent Mind? And maybe I want to focus on the term emergent, because we all think we know what the mind is, and we’ll come back to that. Right. So, emergent, what does emergent mean? The The definition of emergent is a property that’s present in the whole, but not in the constituent parts. So, if you look at water, the wetness of water, individual molecules, or even up to six molecules of water, don’t show the property of wetness. So, what we perceive to be wetness is actually because of hydrogen bonds in water. So, water is oxygen, one hydrogen, one hydrogen, and this hydrogen will make a hydrogen bond with the next oxygen. So, because of the hydrogen bonds, there’s a sliding motion of sheets in water. So, while water molecules don’t have the property of wetness, water has the property of wetness. So, this is emergent in the sense that the constituent parts don’t have wetness, but water has wetness. Another beautiful example of uh emergence, which really profoundly affected me, is in ants. So, the If you imagine a train of ants going from their nest to, let’s say, a food source, and they’re going in a line. And they go to the food source, and then they come back. So, imagine that you put an obstacle. And there’s a short way around the obstacle, and there’s a long way around the obstacle. And what you’ll see is that initially, half the ants will go the long way, half the ants will go the short way. But in about 15 minutes, about over 90% of the ants are going the short way. And in my classes, I often ask, “How is this happening?” And their guesses are, “Oh, maybe there’s a queen ant, or maybe the ants are communicating about this is the shorter way, take this.” And the answer has nothing to do with it. The answer is that they are simply following the shortest pheromone trail. Ants do two things relevant to this conversation. They lay pheromone, and they follow the strongest Pheromones are chemical signals. Pheromones are chemical signals. The ants don’t have a conception of the shortest path, but the ant colony is able to find the shortest path around obstacles. So, emergence is ubiquitous in nature, and it it suggests that properties arise at levels and it’s possible that the underlying parts don’t have the property, but the whole does, right? So, uh fluids uh behave or or gases, the molecules that make heat are not themselves hot or cold. It’s It’s a temperature is a temperature heat are emergent properties of the motion of molecules. These are examples of uh something arising without it being ingrained in the parts. Yet, uh when we say emergent wetness, shortest path we are clear about what we are referring to that has emerged. Right. But, when we say the emergent mind what is it that we are saying has emerged? We all seem to think we understand this term. What is it that you mean by the term here? What is it that you would want a reader to take away when he reads or she reads the term mind? Great. I think that’s a wonderful place to start. So, when we think of mind, what do we think of? We think of the awareness of an idea or an action. So, if I am I had to meet you today at 11:00, so I planned that my I’m going to take the Uber at so and so time so that I’m not late. So, that was a very It feels conscious, right? So, we who we marry or who we decide where we go to school, these seem like they are we think about things and we come to some conclusions. And so, for many people, mind equates to consciousness.

[clears throat] The book is taking the approach that the activation patterns in our brains that correspond to our thoughts. Don’t really uh uh create actions by being conscious or unconscious. They’re there. Sometimes, if they’re stable enough, it appears that they come into consciousness. But, if we think of the mind as a sum total of our thoughts that cause action in the world, then one realizes that one can think about thoughts in terms of activations of neurons rather than conscious, linguistic, or visual images that many of us associate with the mind. And there’s very, very good reason to think that these conscious thoughts are often stories that we tell ourselves after the fact. So, let me give you a very simple example here. Let’s say that you get up and you get yourself a glass of water. And I ask you, “Hartosh, why are you getting yourself a glass of water?” You’ll say something like, “I’m thirsty.” Which common sense. But, it turns out that this this conscious awareness of thirst is not is not what’s getting you up. In your hypothalamus, which is a part in in your subcortical brain, um there are neurons that respond to the salinity salinity level of your blood. And when the salinity level is high, that often corresponds to a thirst state. So, these are just detectors. Just think of them as detectors. They generate small bursts of electricity called action potentials when they find that the salinity level is in the blood is too high. And over like these circuits get connected to circuits representing action towards water. Okay? So, it’s a conjunction of an aware awareness in the sense of I shouldn’t even say I should say activation related to too little too much salt in your blood and where the water is. And one can take an action without having this conscious thought that I’m taking this action. You know the water is in the refrigerator uh from past experience. So, the feeling of high salinity in your blood is connected to the action of getting up from the chair and going to the refrigerator. And all of this can happen without conscious without anything coming to conscious awareness. So, here and what you’re saying, I think is that that wanting of water, the conscious or the awareness of wanting water Yeah. is not the motivator for your getting up and getting the fridge. And I think as you said in the book, there are experiments that show that the wanting comes after you’ve already taken the It can. It can, absolutely. So, let’s let’s actually talk about two or three of these because you’re very wise to start here because many of us, including me for the first four decades of my life, maybe more thought of the mind as what I have access to, which is conscious awareness. Um a lovely paper by Nisbett and Wilson in the ’70s entitled uh Telling More Than We Know. They did a very simple experiment. They took four stockings and asked people to pick their favorite stockings. The stockings were arranged in order, 1 2 3

And from decision science, we know that people pick whatever that they saw the last, they’re more likely to pick. So, the percentage of choice was 20% for the first one 50% for the the one and the remaining spread between the second and third. They asked people, “How How come you picked the last one?” Somebody said, “Oh, the color is slightly different. It’s the texture is slightly softer.” Meantime, the stockings were identical. Right? So, the process that caused them to pick it had to do with attention uh towards the latest item. But, we come up often with stories that explain our actions. Uh another very salient uh group of experiments were done in the ’50s with split-brain patients. So, uh we have two hemispheres in our brain, and these hemispheres are connected with a corpus callosum, which is a bundle of nerves. So, people who got epileptic fits, um the doctors would sever the uh corpus callosum. So, you have two hemispheres that are not talking with each other. So, it’s possible to show the right hemisphere Most people, centers of language are in the left hemisphere. So, it’s possible to display uh messages that are not processed by linguistic centers and don’t come into awareness. So, you might flash a message like, “Start crawling on the floor.” And this comes to the person’s right brain because the experiment doesn’t allow that message to be seen in the left brain. So, it comes only to the right brain. But, the right brain processes that information, and and the person starts crawling on the floor. And you ask them, “Why are you crawling on the floor?” And they they don’t have the awareness that I saw this message. You instructed me to crawl on the floor. So, what they say, they don’t say, “I don’t know.” They say, “I think I dropped my keys.” Or you say to them, “Go to the bathroom.” “Why are you going to the bathroom? You just went.” “I think my hands are sticky.” Right? You have people who are trying to commit suicide with one hand and prevent the suicide by with the other hand. So, um these awareness these actions in the right brain are happening, right? I mean, this is a part of the mind. The mind is acting. It’s crawling on the floor or it’s going to the bathroom. There’s an action. There’s some cognitive states that led to the action. The explanations are actually wrong. In in cases like ours where we have an intact corpus callosum, the the explanations for our actions are much more tightly plausible. Right? So, we won’t If you did this to me, I would say, “Artoosh, you just showed me a sign. I’m trying to be a good participant and so I’m crawling on the floor.” But, that’s probably after the fact. Um unless you know, unless we start acknowledging the fact that a lot of the thoughts that create action, acting in the world, may occur underneath consciousness. Um we’ll have a very incomplete picture of the mind. The proposition here is that these activation patterns in brain cells are thoughts. Sometimes they come to consciousness. And they are If they’re stable and long-lasting, they’re more likely to come into consciousness. But, sometimes they don’t and many times they don’t. Either way, they are thoughts. So, by mind, what we mean in by the word mind is patterns of activation that lead to action. And it’s not a commitment to conscious awareness. There is no There is no requirement that this make conscious awareness. There’s also not a requirement that it does not make conscious awareness. But mind for us is patterns of activation in the brain that lead to actions.

[snorts] So here, I just just want to elaborate on this. You’re using two terms. One is conscious consciousness and one is awareness. Are they the same thing? Are they related things? Are Great. So that people debate this. I was using them to mean the same thing, awareness. But consciousness also has an added quality that philosophers refer to as qualia, which is like the smell of a rose smells like something, right? Pleasure is different from pain. The there’s a subjective experience of a taste of chocolate, which is very different from the taste of onions. So we have all these experiences in the world. So when people think of consciousness, they are talking both of these experiences as well as explicit awareness of the the thoughts that we have. And they’re they’re both both under the consciousness umbrella. Yeah. And more or less this image on the cover. Yeah. Is this what it is talking about? Yeah, that’s that’s the idea, right? I mean, I can’t claim credit for that. Freud came up with it. So Freud, right? Freud gets a lot of so this is a slight tangent, so you can stop me anytime, but Freud gets a lot of like oh, how how can he be such a fool about so many things? Freud was wrong about a lot of things. But Freud was the first person who intuited that a lot of what we do is happening beneath awareness. And he came up with that iceberg image. And of course, in this book we’re presenting thoughts as activation patterns. Many of these thoughts don’t make it to the level of consciousness. Some do and that’s the top of the iceberg. And many don’t and that’s the bottom of the iceberg. [snorts] And so I want to just clarify the terms and there’s a third term here sitting on the cover, which is associated with this, but is not the same. And this is how intelligence arises in people and machines, and I’ll take up the whole thing later, but intelligence as opposed to mind or awareness, what do you mean? So this is a deep question, and the at one level the answer is very simple. The book is called the emergent mind. And so to have a subtitle that says how the mind arises in people and machines is a That’s a simple answer. But let’s use this opportunity to think about what is intelligence. Intelligence is flexibly responding to context. So if an organism can flexibly change its behavior to changing context, that is intelligence. What does intelligence entail? Intelligence entails some internal representation of that context. For us it’s in our brains. Um an octopus, by the way, has eight brains in eight legs and a central um brain, also. And but what is what are we doing or what is the octopus doing? It’s responding to things in its environment. If it comes to a uh to a threat, it’ll it’ll find the best way to get away from the threat or if it has a opportunity to eat. So it’s constantly change it’s constantly changing its behavior to respond to the realities and the constraints of the of the context. And I think that’s a that’s a good enough definition of intelligence. Like one of the things Wittgenstein, the philosopher, said it’s a myth to think that these things have clear definitions, right? Like if I asked you what’s a chair, you might say, well, chair is something with four legs you sit on. Well, first of all some chairs have three legs or some chairs uh don’t have legs at all, like those bean bag chairs. Um chairs, you know, whatever definition we come up with, like there are exceptions to it. Like, what’s a game? Game is something two people play. Oh, really? No, one person can play a game. Um game is competitive. Game is with points. No, not really. So, um concepts don’t really follow classical definitions. And here we are talking about the mind in a particular way. We are saying the mind is these activation patterns that allow us to respond and act in the world. And intelligence is our ability to flexibly respond to these changing context. Um in that sense, the mind is producing intelligence, right? And intelligence can be without consciousness. Gosh. Um So, it’s too early to talk about large language models, but but in this context, let’s let’s do that. Um there are two years ago, when somebody used to boast about large language models, they’d say, “Oh, they’re getting 1,500 or 1,600 on the SAT.” Um then it became 1,600 on the GRE. Then it became that they’re acting like a graduate student-level researcher. Now, um there’s acknowledgement that large language models are helping solve unsolved previously unsolved mathematical problems. Um large language models are performing better than many doctors, many lawyers, you name it, they’re doing it. Um they’re flexibly responding to changes in context, and in its case, the context is what you tell it. Um so, if you tell a large language model that I approached the bank with my fish fishing pole, and you ask it, “Can I withdraw money from this bank?” It’ll say, “No, this is likely a river bank.” Right? So, it’s it’s very context-sensitive. Um 2 years ago, it couldn’t do that. Um 2 years ago, if you asked the question, “There are 13 people in a room, what is the probability that uh two of them at least share the same birthday month?” 2 years ago, it could not do this question. Now, it can easily do this question. So, large language models, at least for me and for many purposes, large language models are intelligent. Meaning that they can respond to context flexibly. Um I don’t believe large language models are conscious. Certainly, what we know about how they perform doesn’t require any attribution to consciousness. They are neural networks, and we can talk about what neural networks are, but they are basically, um simple processing units interacting with each other. And the intelligence of a large language model emerges. None of those units are um intelligent by themselves, but it emerges in that interaction. And it does not depend on consciousness. So, and I promise not to show you the cover again. I’ll put the book down after this. [laughter] But this the subtitle here, “How Intelligence Arises in People and Machines.” And one part you’ve already answered, that it is clear from here you imply machines have intelligence, LLMs. The second is to me an even more interesting claim, that the intelligence arises in similar or same ways, because it is how intelligence arises. And that is the claim I want to take you to. That the substrate or the explanations of how the mind emerges to you is the same in either case. Yeah, Hartosh. Um you know, I’m primarily interested in the human case. Um I started doing neuroscience uh after the working on mathematics. There’s a slight personal uh sort of digression, but we both studied mathematics. Um We both never finished our PhDs in mathematics. Um and I think we both could have, but we chose not to. Um so the initial interest uh in mathematics for me was really about what happens when we think mathematically. What What is going on when we think mathematically? And this was a human question, and we explored some of these questions in our book, A Certain Ambiguity. Um and there we borrowed ideas from mathematics that you start with a set of axioms, and you logically act on those axioms to produce thoughts, right? This is This is what I grew up with. I thought that intelligence is starting with some axioms and doing symbolic logic on them. And where does this symbolic logic come? Evolution has granted us the machinery to know that if A implies B and A is true, then B must be true. This is uh Socrates is a man. Uh all men are mortal. Socrates is a man. Therefore, Socrates is mortal. We have the intelligence to uh machinery to do this. Um many people used these metaphors to think about artificial intelligence. So, for about the We’ve been trying to build intelligent machines for the last 100 years. Probably for the last 70 of those years, we built them basically with if-then-else statements. If this is the input, this is your output. Else, if this is your input, that’s your output. So, these were rules. This is called sometimes called rule-based AI. Sometimes it’s symbolic AI, which is you represent variables in the world with symbols and then you perform logic that doesn’t depend on those symbols to come up with conclusions. Um but you know what? Uh people are faster to answer questions about their grandfather uh such as all men are mortal your grandfather is a man therefore gran- your grandfather is mortal than they are about the same logic structure about entities that they don’t know about, right? So make up some words. Tribologia uh is a man all men are flabogeia or something. You make up those words, the same logic doesn’t work. People make mistakes and there’s many, many cases of this. So AI started with these these ideas that look, it’s rule-based symbolic manipulation the way we do in mathematics. And I uh for many, many reasons uh was was very doubtful of it and in the human case came up with the understanding there has to be a better answer. And this I don’t think we’ve talked about this. This really does uh relate to the days we wrote uh a certain ambiguity. And and what I realized independently and what many other people had realized before that this cannot work and it led to what is called the AI winter. Where billions and billions were invested in building intelligent machines with the if-then-else logic didn’t go anywhere. And finally people said, “How do humans do it?” And one of the best answers at that time uh was with neural networks. Which shockingly to me I saw neural networks before I understood uh their application to AI. It was shocking to me that people thought neural networks are like, oh, you can use it to understand the mind, but there are other theories as well. So, there are theories sometimes called Bayesian approaches, which um postulate that there’s an objective function that guides our behavior. The these are not if-then-else theories, but these are theories that assume an objective function such as utility maximization or uh error correction, right? So, so that we we can minimize errors. So, the neural network approach was this, look, there’s interacting units like we have interacting neurons, and let’s see if intelligence emerges. And people start making these machines, and at first, if the machines are small, the emergence doesn’t happen. Like you need a certain amount of water for wetness. But if you allow enough data in enough of these units arranged in a certain way that learn to make connections, I will talk about connections, um all of a sudden, intelligence emerges in these machines. And this this is the key insight that the machines that we finally got to show [clears throat] intelligence, I think we got them because we relied were inspired by neural architectures in the brain. Our neural networks are called brains. Neural networks, meaning they’re brain cells called neurons that are connected with each other, and we borrowed that idea to make neural networks in machines. Now, machines don’t have neurons, but neurons do very simple things. They they either fire fast or they fire slow. So, they go bing bing bing bing bing, that might be a fast. Bing bing bing, that might be at rest. So, that rate is called activation. So, that’s one number. When you’re saying they do it, there is uh passage of electricity going through the brain. Can we break down a neural network in terms of what it is? So, when we are talking of activation, what is being activated, what is being passed on inside. So, artificial neural networks, you know, they they come up with this idea that the brain has these networks of connected neurons. And can we use that inspiration to make artificial neural networks? So, as I was saying that the neurons have this number called activation, which is how quickly they are sending electricity from one end of the neuron to the other end. And it could be very fast, so that’s high activation, or it could be slow, that’s low activation. And so, that’s one property they took, which is instead of acting having neurons, you can just imagine that there’s a circle. The circle, you don’t draw a neuron, you use a circle. And you don’t have electricity, you have a number that stands in for how fast the neuron is the neuron is either activating or not. So, they took that idea, and neurons also connect with each other. So, if I say green, you will say trees. Trees. So, that’s because your neurons that um are activated at the sound of green are connected to neurons that represent the thought of trees through previous experience. And that’s why when I say green, you say trees. Um of course, if you’re in a political context, if I say green, you might say party, right? And so, that the political context is a added input, um which will make you flexibly respond in your output. But the idea is that in our brain, if you abstract away some of the details, all the brain is doing is it has units that are activating, and they are connected to units, and if they’re connected to those units, they make their activation change. Right? So, when before I said green, you didn’t have any activation in trees, but as soon as I said green, you got activation in your units representing trees, and that’s because you can imagine connections being like pipes almost or lines so that this activation influences this activation. So, in a neural network you have a unit that gets activated and if it’s connected to another unit, it may activate that. And they took these two ideas and they said that look, we can make software which doesn’t have physical neurons, but we abstract out everything and we have these units that will keep track of their activation and some units will be connected to other units and those connections will represent with numbers in a matrix and that’s a neural network. And that approach produced large language models. And the earliest large language models, the people who developed them were shocked at their ability to have conversations. There’s no if then else logic in a neural network. There’s no if the user asks about the weather um talk to them about clouds or wind or temperature, nothing. Um we can talk more about what these things are, but they’re emergent machines uh that are neural networks just like we are emergent organisms that have neural networks. And so that’s the subtitle. And So, in some senses these core properties stripped of all the biology still retain enough to do almost similar things, not the same, but similar things with sort of similar results. Yeah, this is this is a core observation. Um and it turned out that the pioneers who invented neural networks and started playing with them and started popularizing them and they were popularized they were started in the 40s. Um they were really popularized in the 80s by David Rumelhart and uh J. McClelland, the the author of the book. Um And what these scientists realized is that if they don’t abstract if they abstract too much, that’s not useful, but if they abstract too little, there’s too much detail. So, there was almost this intuition um of, you know, if we just get connections and activations and simple processing units, we’ll be able to make progress. And they were right. Now, maybe we should introduce other things um that are present in brains if we want our neural net our machines to have more functionality. Uh but that’s the These are the two things they started with. And that’s led to the AI the age of artificial intelligence, which by the way, I liken to the um the printing press or fire, right? I mean, I think that um humans 200 years from now are probably not going to be purely biological. Um we are going to have cognitive structures in our brain. We’re going to have little robots in our bodies that can hunt down disease cell and zap them. Um life is going to profoundly change. It’s unclear whether the change is for the good or the bad. I think that’s up to us and the checks and balances that we put on each other. But definitely the age of AI is upon us and it it’s because they abstracted at the right level. Um and that led to these outcomes. And I think we’ll return to these possibilities that you’ve raised of the future, and I think it interests a lot of people, but I think they need to understand actually how we get there and what has happened. And so, when you I want to go back to the substrate. When you say a A network representing green and a neural network representing a tree has connections. Great. And they connect to each other. But how in the first place does a neural network represent green or a tree? What does that mean? I Because we have to start with the very Great. Um So, what does it mean to represent an idea? So, imagine that we have three neurons. Um and activations of those neurons are one or zero. Okay? So, the neuron is either got high activation, bing bing bing bing bing, or no activation at all. How many ideas can we represent? Well, we can represent eight ideas. 000, 111, and and those combinations, right? 110, etc. You can count them up. So, here an idea corresponds to a pattern of activation in those three neurons. Now, real neurons or populations of neurons have activations that are not just zero or one. They can be any number, and often we’ll have a convention that they’ll be between a small negative number, which is below baseline, zero might be baseline, and one might be the highest possible activation. So, think of activation as a number between, let’s say, -0.2 and 1. And you have many neurons. And guess what? You can represent ideas by patterns. So, if we have five neurons, the the 10001 represents a particular pattern, and that particular pattern may stand for the concept of green. And another concept like another number pattern like 111 11000 uh might stand for another pattern. Of course, there are many more ones and zeros, as I said, are not uh they’re idealizations. It might be 0.8, 0.3, 0.4, right? It It might be different activations. So, think of these activations as patterns of flashing lights. And a particular pattern represents a particular idea, and a different pattern represents a different idea. And these are in maybe in the same neural substrate, or it may be in a different neural substrate. But, the beauty of representing ideas by patterns is that similar things in the world can have similar patterns. So, it would be really strange if my pattern for sofa was very different from my pattern for my pattern for couch. Those things are often uh basically interchangeable. And it turns out that the brain has this remarkable uh ability and we don’t understand how this ability arises to represent similar things with similar patterns. But, if people are uh following along this notion, just the only thing to keep in mind is that a idea of a thing in the world corresponds to a particular pattern of activation on a neural population. So, this part is in some ways clear that you can use a population of neurons to have several patterns, and each of these patterns can correspond to something some idea something whatever it is. Mhm. But, the intrinsic basic question here is how does a particular thing or a particular idea come to correspond to a to a particular pattern before that pattern starts making sense? That’s great. So, let’s imagine a baby. Baby opens its eyes, and it sees this soft creature who gives it food, right? And what’s happening? The baby is got input coming from its eyes and it sees, let’s say a a round face with long hair. And it has input coming from its nose and it smells its mother. And it has input coming from its mouth when it drinks milk. So, these What are all these? These are neural patterns that are starting in our eyes, ears, or mouth. These neural patterns are traveling to the brain. Nerves carrying electric signals. Exactly. These are nerves. These are not wires, these are nerves, like the optic nerve. By the way, the way the Romans first figured out that the brain is the seat of cognition is because they followed the optic nerve. Before that, they used to think it’s the heart. They They’re looking at where’s the eye going to? And I I often imagine this guy and he’s sort of pulling at the optic nerve and he realizes it’s it’s it’s on the brain. So, the optic nerve is going to the brain. And each just innately connected to some neurons, right? Those neurons start to fire. The nose neurons are connected to some other parts of the brain. Those neurons start to fire. Now, these are innate connections. The baby is born with these connections. But, something amazing is happening. Every time the mother is coming in, these neurons are co-firing. So, they become um connected with each other and they are forming their own pattern now, right? So, the baby through these repeated um exposures to its mother is getting a pattern that corresponds to its mother. And then one day, the father picks it up. The baby says, “Yuck.” Because the smell is different, the the the eyes are different. And then the pattern is going to be different. So, the pattern is a result of initially the result of innate connections from our sense organs to our brains and the connection patterns there, and are then developed by experience, and we learn to expect different things, and we learn that things co-occur in a certain way, and and that’s how these patterns arise. So, and if I understand you correctly, there will be a pattern, let’s say, for mother, Yeah. and a pattern for father, Yeah. and if and uh maybe I’m going by conventional usage that it is the mother that is a primary nurturer, and that could vary, but let us assume that. Yeah. So, the pattern for something like milk Yeah. will also be formed, Yes. and it’ll be more closely associated with mother than, say, father. Yes. So, and these associations, do they work one way or both ways? Do they milk also evoke some associations with mother in itself? Well, lovely. So, what’s your association with red? Uh the communists. The communists. Hopefully political. [laughter] The game took me to trees, but Great, great, so so okay, so red is suddenly communist. Now, let’s constrain this, right? So, you’re driving, and you see a red light, right? And that the the connotation there is stop, right? So, it’s not it’s not true that one thing in the world is connected to one thing unidirectionally or even bidirectionally. Many things are influencing each other, and stable patterns are arising. In the context of seeing a parent, milk is going to be traditionally more connected to the mother because of co-occurrence. And when things co-occur together, their representations tend to get connected, and the connection is bidirectional. But, my point is that it’s not um it’s not deterministic in the sense it’s not required I should use the careful word. It’s not required that the thought of milk will always make you think of your mother because that depends on context, and other things will also get connected to milk. But, it is true that if if it’s uh which parent, the question which parent it’ll always be because it co-occurs more with the mother than the father. So, it’s going to be a tighter thing. For now, if you’re following along in the conversation so far, um just think of things in the world are patterns of activation in neural populations, and some patterns tend to co-occur with other patterns, and they get bound, and they influence each other. And so, that if I say red and you say Calvinist, that reveals a pathway that you have in your brain. But, again, these are context-dependent. Right? And and I’m jumping steps and I’ll come back again, but this also suggests that when you say these associations form, and they form through observations, yeah, that even what we call biases have taken place due to our own experiences, and we have to be aware or conscious of these things in some ways, and we’re jumping ahead, but [clears throat] Yeah, I think this is a central point. Um and you already, you know, uh like if I always see milk coming from my mother and never from my father, if I hear footsteps and I’m hungry, I am hoping um or expecting my mother, right? It’s already a bias, you know? So, what we call bias is actually comes from the same machinery that allows us to form concepts in the world. And we would be lost without concepts, right? So, when I say tree, you expect a certain thing. You have a concept of a tree. You don’t expect a tree to typically come out of where it is and charge towards you. But, that is possible with a bus. You know this through repeated exposures to a tree and repeated exposures to a bus, you make concepts in the world. Now, if you see certain types of behaviors uh with a certain group of people, it’s the concept you’re forming associations, right? Often with sparse data. The reason we call these biases is because humans will tend to form them not just with the sparse data, which is already bad enough because we make connections very quickly. Sparse data we see we see three things and we say so-and-so race are bad drivers or so-and-so race are criminals or so-and-so race are smelly or whatever, right? We do that with very sparse data, but guess what? There’s also these emotional circuits about group formation and group identity, which are powerfully associated with um chemicals called neuromodulators that make the activation patterns even stronger than they are. And biases come about from us associating things in the world supercharged by these uh ancient instincts of of tribalness. Um we would be lost without our machinery to form concepts. Biases are one kind of concepts. One kind of concept. Happily, we also have machinery that prevents us from acting on activation. Not magically, we can talk about this. The These are yet other controlled activations. So, in the game of Simon says, if I say Simon says raise your hand, then you will. And if I say raise your hand, you won’t, even though raise your hand is priming the act of raising your hand. But, you’re able to tamp down that activation because of this other circuit, which is active, which let’s call it a goal circuit, which is active and uh tamps down the activation of raising your hand. So, how do we end up with these These seem to be higher-order circuits because the act of following a command of raising your hand seems to be almost instantaneous, built into whatever representation we have. And then there is another representation, which is modulating it. Yeah. And where does this hierarchy construct itself, and where does it come from? Right. So, the cheap answer is that it comes from our extended cortex. So, the cortex is the outer covering of the brain. We have a cortical sheet, which is about this big, the newspaper size. If you ex- If you extend it out, now it’s folded in in many crevices. And the reason if it’s fo- if it wasn’t folded in, our heads would be this big, and we’d never get out of our mothers’ birth canal. So, it the cortex is able to have all of these representations simultaneously. And specially in the frontal cortex, we are able to have goal representations that can with attention stay active, and we actually learn in life uh through a a a network that we are born with, but we learn how to use these activation to interact with other activations that are relevant to it. How do we know? There’s no manager up there. How do I know that in the context of putting sugar in tea, I should have a health goal and not a goal of getting more exercise or not a goal of writing a paper. We know because the bottom-up patterns associated with the tea are activating the goal. The goal once activated is folding back on the action of the tea. These are These exist in the cortex and they get um a boost from our motivational machinery. Um and it’s our particular neural architecture that allows us to have both quick action and responsiveness as well as the ability to prevent such quick action and responsiveness. So, in some senses, so the idea of thinking or say higher order activities which exercise control, uh do they follow in life itself an evolutionary chain? You can have life which is just about seeking food, avoiding danger, and subsisting exactly at that level without these higher goals coming in to curtail one or the other? Most of life, I would say, is like that. Um I have a dog who you met. Um He has the ability to curtail some of his actions, but much less than ours, right? I mean um there are animals that are very driven by instincts and um drives and have very limited capacity to stop that machinery. So, if a spider is weaving its web, it’s informed it’s intelligent because it’s informed by the context of where it’s hanging, maybe temperature and pressure, but it’s not able to regulate its behavior or control its behavior once a particular context is set. It’s weaving its web. Uh we can control that behavior and that’s because neurons can be organized in different numbers and different architectures and certain architectures give rise to abilities that are not possible in simple architectures. The simplest architecture is two units. Let’s say one unit represents red and the other unit represents communist and red has a connection to communist. Every time I say red, you will say communist. That’s the simplest circuit and that is utterly unflexible. But um many many worms, many bacteria perform very um in some context they perform very predictable sort of behaviors that don’t have this cognitive aspect of being able to control them. Even though life is extraordinarily context sensitive. These higher order controls or our perception of them, as you say, sometimes those are stories we tell ourselves after we have done something. But they also allow us to form sort of whole stories of ourselves as individuals, as people doing things, receiving things, interacting with others. Uh is this fully possible without consciousness in the sense would a dog have a similar story? And at what level does consciousness enter the picture as a necessary object at all? Great. So um what is the self? We have this notion that the self is a internal controller um that tells us what to do. There’s a very obvious problem with this notion, which is who’s telling the controller what to do. Who’s the controller’s controller, right? So, if we don’t think that the self is a deep manager inside our brain, the question is what is the self? And your question is does does the self need to be conscious? Um So, here’s my proposal to to you, which is the self is an aggregation of patterns of behavior in various contexts. So, suppose I know you, so you you are curious about um the world. So, you know, you do mathematics for fun. So, that’s a aspect of your self. And you observing yourself, you observing your own behavior are going to know things about your likes and dislikes by observing them in the world. Nothing to do with consciousness, right? Nothing to do with consciousness. But you will get a a notion of yourself, just the way a dog might get a notion of itself, such as it likes to chase after black squirrels and not uh brown squ- I used the word like, which is a loaded term. It chases after black squirrels, but not brown squirrels. It has representational structures of its own behavior. Nothing to do with consciousness. It mean some of this may come to consciousness, right? But if you have this pattern of activation related to yourself, it can be highly influential without being conscious. It turns out that um activation patterns that are stable and um strong are very likely to come to consciousness. And we have these chemicals in our brain called neuromodulators, such as dopamine or serotonin. And we have very likely um these patterns of the self are going to come into consciousness because they’re often boosted by these neuromodulators. Why? Because through our evolutionary past, it’s been important to have notions of identity and how we relate to others. This is a part of our evolutionary past. And sense of goodness of our own self is is rewarding, has been rewarding evolutionarily. And so these are supported by neuromodulators and they boost these activations. And so therefore activations of self are much more likely to come into consciousness. By the way, it’s not the consciousness that is making these things influential. It’s the activation. It feels like it’s the consciousness. But it’s the activation pattern and this pattern can be influential with or without consciousness. There’s no requirement that it folds back into the system only from conscious experience. And the thing is that it’s very hard to come up with an a functional role for consciousness that is not possible in a system without consciousness. Lots of people spend their careers thinking about this and there are many hypotheses. Um some people think consciousness is a way to globally broadcast things that are important in the current context. And these are reasonable things, but we don’t know and the reason we don’t know is because no matter what people come up with, we can come up with a design of a network without resorting to consciousness that has a similar property. So Gautam, you mentioned neuromodulators. Where do they fit into this large picture of neurons and activation and units? Great. So Artosh, so far we’ve got a neural network which could be biological, which is called a brain, artificial, which is a large language model. And we have made the point that listen, they’re simple processing units. In the brain they’re called neurons, otherwise they’re called units that are connected with each other, that take some input and through the network that input flows, causes activation in all these units, and some output comes in. So, for example, if you see a bus coming at you, you’ll get out of the way. If you’re If your neurons for thirst are firing, you’ll get some water. Those are examples of inputs and outputs. So, neural networks have a business-as-usual sort of way, but in some cases, it’s really important to mark significant events with something that takes the neural network from out of its business-as-usual processing into a new level of processing. Maybe it’s really important to learn something new. Maybe it’s really important to pay attention. So, if a particular berry is very tasty, a monkey should very quickly learn to pay attention to the berry. Well, what does it mean to pay attention? There’s no inner manager. There’s no inner controller. So, what does it mean to pay attention? Well, it means to have higher levels of activations for that berry. That you want that pattern to be stronger. How should you get that? Well, you go away from business-as-usual, and the chemicals in our brain that cause you going away from business-as-usual are called neuromodulators. So, there Let’s give some examples. Many of your viewers would have heard of dopamine. Dopamine acts both as a neuromodulator and as a neurotransmitter. Neurotransmitter is just the thing that causes communication between two neurons. But, neuromodulators flood the whole neural network and make the whole neural network behave differently. So, activations are can be more salient, and learning can be faster. That’s what neuromodulators do. We, biological creatures, have neuromodulators. LLMs don’t have neuromodulators. But, I think it’s important to sort of because I’ve said the word dopamine, many people think incorrectly that dopamine is the reward chemical. How can a chemical be a reward chemical? This because if you if you allow a chemical to be a reward chemical, you’re saying that it affects consciousness, right? Because you get into pleasure and pain. Really, you can get neuromodulator effects because they make action more urgent. Because they’re amplifying the activations. They’re increasing the strength of the connections. So, once a neuromodulator is present, like dopamine, it’s the it’s the it’s involved in circuits that cause persistence and uh opportunistic persistence in the pursuit of something that satisfies a need or a goal. And dopamine makes that network active. And that’s why it’s better thought of as the wanting or seeking chemical than the reward chemical. Reward is you have a good meal and you feel like sleeping, right? I mean, that’s reward. That’s not that’s not dopamine. Um Dopamine wants you to have that good meal. Dopamine wants you to pursue that thing, right? Actually, serotonin is a chemical that’s more associated with the seizing of pursuit. And that’s why when people take SSRIs for when they’re depressed or have major depressive disorder, serotonin will cease it’ll give the networks that are associated with calmness or the cessation of any kind of pursuit, they up amplify those activations. The problem is that many people who take these SSRIs report a lack of motivation, and that’s not surprising because while the the relationship between serotonin and dopamine is complicated, at the first approximation, when dopamine is high, serotonin is low, and when serotonin is high dopamine is low. So, if you flood the brain with serotonins, which is what SSRIs do, then motivation circuits are um dampened Is there because I mean equally important as activating a rem- the desire to eat a berry. Yeah. Avoidance of a snake or poison. How is that motive? Boy, that you better learn if you see a tiger at the watering hole not to get go to the watering hole and you better not require thousand repetitions to learn that because you’ll be dead. Um so, the brain has to have ways of one-shot learning where learning occurs nearly instantly and um that kind of learning is supported by neuromodulators. Um Often things that we need to learn quickly are associated with high activation. We think that it’s the conscious elements of that high activation such as I’m scared of snakes. No, the the conscious feeling of fear is accompanying these circuits that are causing you to withdraw, avoid the slithering motion in the grass. And one doesn’t need to evoke the feeling of fear or the feeling of even pleasure in pursuit. Yes, these are they they co-occur but they need not be the cause. Very interestingly, um there’s a story of a patient who um used to experience involuntary uh orgasms. And for her, these sensations were not pleasurable. They were painful. It’s the same and you know, medical doctors verified that it’s the the set of processes occurring in the body, but her things were involuntary. So, it was seen it was associated with these things are intrusive, these things are uh stopping her from doing what she’s doing. She experienced these as painful. So, my point is that these ideas of pleasure and pain, they seem to emerge out of this underlying machinery, uh rather than drive the function of the underlying machinery. Um so, it’s these neuromodulators. And one can imagine neuromodulator-like things in machines that disrupt businesses usual, that speed up learning in certain in certain context. And one does not need the feeling of motivation or the feeling of emotion to uh have machines that act with urgency in certain context. We will get to LLMs and machines maybe in the second part of this discussion. For now, I just want to take you back to this bit. We have neural nets, you have activation, you have patterns, you have labels, you have higher-level units, you have neuromodulators. Is that [clears throat] it? Is that everything in terms of uh telling us how the brain works in terms of its units? How does that explain and this is the question you’ll get all the time. My idea of wanting to paint something I see, to create something I see, my sense of belief and religion, whatever we see as the higher things of a human being. And people react very badly to the idea of usually to them being reduced to activation. Right. So, [laughter] um we’re getting to things that spark a lot of debate, understandably. [clears throat] I want to be clear, you know, um Kahneman, Daniel Kahneman, um in his book Thinking Fast and Slow, uh Daniel Kahneman was a Nobel Prize winner, and he uh came up with these ideas of two systems of thought. System one, which is associative, acts very quickly, and system two, which is deliberative, and um slower, and seems much more to be associated with the higher functionings of an organism. The story I’m trying to tell you is that there are no two systems. There is no system two. There’s only a system one, and that system two-like characteristics emerge from the interactions within system one. And this is certainly um not an obvious statement. It may not even be the true statement. I’m but this is what I believe, uh based on the idea that one can trace the emergence of system two-like capabilities from the neural network, which is not a system two thing. There is no deliberative body. This is all associations. Everything is activations of various kinds. Now, the the argument that you’re making, which is, are you sure that uh poetry and love and painting and mathematics come from these rather dry activations flowing through connections? Um I believe so. And I think it’s a bold claim. It’s a big claim, and it requires big evidence, and this is the topic of computational neuroscience. This is what This is what I want to spend the rest of my life thinking about, which is, is it possible that all these things that for us define our humanity come from this underlying neural neural network? Um So, when it comes to let’s take let’s just take mathematics. We can trace certain mathematical ways of thinking as emerging from neural networks. We don’t understand the full detail of how advanced problem is solved. But the fact is that large language models are showing this behavior. And as far as we know large language models are system one models. They’re associative models. Um now is it true that our desire to paint something or a desire to fall in love is immune from this is somehow some other system. It’s so tempting to think so. A quick quick diversion, Descartes, um the French philosopher mathematician is walking in a garden and he steps on a stone. He steps on the stone and a statue moves its hand. It moves its arm. And so Descartes the story he writes about this. He goes to the gardener and they have a conversation. This is in the royal Versailles something. And he asks, “What’s how come when I stepped on this stone the the statue moved its arm?” And the gardener explains that there’s a system of tubes. And he the stepping created a pressure which created turbulent etc. So, Descartes says this is great because he’s got a mechanistic theory of action. And he says that this is how um humans work. And when we are too close to the fire Descartes imagined there’s a small tube that runs from our finger to the brain. It compresses the fluid pressure and the brain reacts to that fluid pressure and gives a signal to go back. Descartes was wrong in almost every detail. But it was a brilliant way of being wrong because Descartes was coming up with a mechanistic answer. Um and this was the first attempt to explain the mind the way we explain our stomach, right? So, if somebody says, “How do we digest food?” We come up with ideas of enzymes and acid being secreted in the stomach. We don’t say this is this is some deep magic. But, in the mind, especially when it comes to these higher-order things, we do some say it cannot just be this. There has to be something else. And that’s what Descartes did. So, Descartes came up with what is now known as Cartesian. Cartesian means referring to Descartes and dualism. Cartesian dualism, which was Descartes’s idea that the simple stuff of the mind is system one, but anytime you get to love and painting, that’s that’s a different system. And of course this this led to ideas of cruelty towards animals, for example, because Descartes thought they’re only doing the mechanistic thing, but we humans are touched with this magic stuff. Soul or whatever that gives us the ability. But, I think there is a very to me compelling notion that we can understand all our behavior in mechanistic and deterministic ways, and we don’t need to resort to ghost in the machine. And wherever I look, whether it’s emotion, whether it’s motivation, whether it’s mathematical like thinking, I am continuously impressed by the flexibility of a non-conscious machine-like architecture to give us that kind of behavior. Where do you think our own and I don’t think that’s necessarily a subject of scientific research, but you face it a lot. This notion of soul or magic or now it is reduced to a more practical term which doesn’t have other worldly associations, but our creative self, for example. Our need to protect it from reductionism, not to see it tainted by matter, material, and simple explanations come from. Well, I think it comes from this idea that the physical world is ordinary, even dirty. And that our mental world is our connection with God, however people perceive God to be. And my response to that is we are processes. Like photosynthesis is a process, like water running down hill is a process, we are processes. Uh, we are processes that unfold in the universe. We are amazing processes of amazing complexity, but we are processes. So, where does that leave us? We are processes like the trees and the stars. So, we are no more than the trees or the stars, but we are also no less. And if you sort of reframe this as the universe being I am so utterly beguiled by the complexity and the the the just the amazing fact that the universe exists, and that there are regularities in the universe, and there are patterns in the universe, and that complexity emerges, and the complexity somehow can look upon itself. Um, that that happens. That I think there’s no downside in being a process in the universe. We are part of the universe. For me, this is a idea of solace, um, rather than uh, that we are just a part of the universe. Are you kidding? The universe is amazing. We don’t understand how it came to be. Um, we don’t understand what, at a deep level, what constitutes mass and matter. I mean, we do fundamental things we don’t understand. So, what is the problem with being a part of the universe? I think it’s great. We’re the only part of the universe, as far as we know, that can look upon the universe. And we are a fantastic process, but we are a process likely a deterministic process. The flip side of this the creativity is the argument that if we are material matter activation then what happens to ethics, morality, law? What stops us from murder, mayhem, etc.? Where does control over humanity of over what we will term our base urges? Is it somehow seen those are material or higher urges are creative? How do we balance all that out? Right. So, um the traditional answer to this is that people who do the good things are good and people who do bad things are bad. And the bad thing people should be punished in whatever way. Different cultures have different ways of handling this. In Hinduism the the story is that if you’re bad then you’re born as a bad life form, etc. So, the neural networks perspective on this is that we are processes that are functions of our connections, the one that we are born with and our experiences, meaning the connections that we form. So, when we have experiences we form new connections. So, a person’s behavior is a deterministic consequence of his or her innate connections and his or her experiences because neurons are neurons. They’re firing. They’re sitting there and firing. So, what sense does it make to say this person is good or this person is bad and should be punished? In my opinion, it doesn’t. This is not to say that we condone any behavior because we as a society must agree on things that civilization that we want to protect and therefore we want to encourage certain behaviors and we want to discourage certain behaviors and therefore we in a state of non-emotional, non-motivational, business as usual get to form checks and balances that encourage people to do pro-social things and discourage antisocial things by punishing antisocial things and rewarding good things. This is the point of culture, I think. This is the point. It’s a utilitarian argument. But, I am not impressed by this idea that some people are good so they do good things because that that’s not a helpful idea. And in some senses, this view of activation and how we reach certain places through experience, through in it may give us better means of regulating behavior in society than is ordinarily done through just reward or punishment or whatever we have come up with so far. Absolutely, Hitesh. I mean, I I love your question because I think the same way. But, um I I I certainly think that if one sees one’s activations as processes, one is much more likely to lead with kindness. Cuz I genuinely think that your processes are pretty similar to my processes, um although you’re a better mathematician, but qualitatively, they are similar processes. And I think that that give is more likely to forgive others and forgive ourselves. And if one takes the attitude of these being context-driven processes, one can do things like plan in advance to put checks and balances. So, if I don’t want to eat chocolate, I’m not make I know that if there’s chocolate in a moment of hunger, I will have the chocolate. So, you know what? At the cold moment of being in the store, I won’t buy the chocolate. That’s a check and a balance on myself. We can put checks and balances on each other. So, if you’re speeding, you you should somebody should prevent you from speeding because that causes behaviors that are undesirable to us as a culture. I For me, the point of uh punishment is not ethical. The point of punishment is to create cultures with appropriate checks and balances. In what what makes American universities great or at least for a long time they have been great is this idea of mentorship where you hang out with a person and you learn with them and merit is rewarded. This is a culture, right? And I would argue that that culture is going down because some of the checks and balances are going down. We have to put checks and balances on each other, on our machines. And that’s the point of building culture. And in the same context and I want to end with this before we do a part two on LLMs, which is a story in itself, connected to the story, is through the back door the third sort of elephant in the room that is brought into this argument is that none of this explains consciousness. So you’ve left something out of the story. So it’s not just creativity or reward or ethics or morality. What about consciousness? The C word, the big C word. Hartosh, we’ve done physics for uh centuries um and we don’t know what mass is. We don’t know what mass is. You know, we don’t know we think it’s it sometimes behaves particles, sometimes this uh maybe it’s a waveform, maybe it’s energy, maybe it’s a fluctuation in a field. We have no idea what mass is. Okay? We don’t know what charge is. We know there’s positive charge and negative charge. What is charge? We have no idea. We don’t we hardly know anything about any um concept at when we ask the question what is it? We we don’t have good answers for it. Does that mean we can’t study consciousness? Far from it. Like we study physics because we we make observations, we we make predictions. In quantum physics, we can make exquisite predictions without having any clue what the underlying substructure is. I don’t know what consciousness is, but does that mean I shouldn’t study the mind? Not at all. I know that conscious uh the conscious field, quote and quote, emerges for certain kinds of activation patterns. Pattern activation patterns that are top-down supported and bottom-up supported that are long-lasting and strong. That enters consciousness, right? Very similar to a story in physics that if you uh throw a projectile, it’s going to take a parabolic shape. I don’t really know what a projectile is, but I can predict the context in which it’s going to act a certain way. I can study the the neural correlates of consciousness. I can hypothesize, although we don’t hear even here we don’t know the answer. I can hypothesize about the functions of consciousness. I can note that the consciousness seems to be a cause of our behavior, and it often is not. I can I can make all these observations about consciousness without solving the hard problem of consciousness, which is the hard problem of consciousness is a simply stated problem, which is how is it possible that activation, which is patterns of electric or patterns of electricity in neurons, how is it possible that those patterns of activation have experiences associated with them? The smell of uh wet rain, which is called petrichor. How is it possible that this this occurs? Yeah, that’s a hard problem. I don’t I don’t know the answer to the hard problem, but I don’t think that that’s a prerequisite to understanding the mind in a mechanistic way. Nor do you think it’s an impossible problem? Yeah, I I So, there are problems where I feel like I know how to start approaching them. Uh I don’t know how to start approaching the hard problem. Um if you force me, I would say it’s some sort of emergence, but it depends on systems. I think in biological tissues that I I don’t even have a slight handle on, but I do not think it’s an impossible problem. Um and I also don’t think that we’re close to solving the problem. On that note, and I think much of this creativity, ethics, morality, and neural networks and activation would naturally lead us into the world of aliens, which is what we will come back to. Yeah. Thank you very much, Golem, for this conversation. It was wonderful. Thank you. Thank you.