Inference Not Prediction Michael Jordan Mlst

TITLE: Inference, not prediction — Prof. Michael I. Jordan on what modern AI is still missing CHANNEL: Machine Learning Street Talk DATE: 2026-05-20 ---TRANSCRIPT--- Nature said that you are the most influential computer scientist.

It exists in the real world. This is a abstraction, but it’s a it’s a real thing. It’s like F= MA. It’s a set of um it’ll make predictions. So if I write down a game just like I wrote down F= MA in some coordinate system, I can now predict what’ll happen. I don’t think we need to see I think this anthropomorphizing of intelligence and understanding all that is not necessary, not appropriate and is is a distraction for many many problems. why say it understands. I think it’s science fiction and I think science fiction is important for society but it’s also at the level it’s being promoted and and and those kind of voices it’s really hurting 25 and 20 year olds. You know these these young folks of whom there are huge numbers are excited about technology and they want to build things that help their family and help their country actually more their family than their country honestly. and they they they see real opportunities in doing that and they’re kind of being told by the leaders, well, we had our fun and we developed a bunch of algorithms. We we did it and we were just interested in the pure, you know, understand intelligence even though they didn’t understand intelligence. They built, you know, grady descent algorithms and now you guys, you can’t do this because it’s dangerous. It’s going to it’s going to wipe out humanity with a with a high probability or it’s super intelligent arrive soon so there’s nothing left to do. That’s in your lifetime. That is so demoralizing. so demoralizing and that thing I think that bothers me the most. I mean the second part that bothers me is there’s no economic thinking going on there. So the current generation is just way too you know there’s not much thought going on not much intellectual stuff. Uh it’s just yeah it’s possible to build it. It’s possible to steal the data from wherever you want to because that’s what the internet allowed to happen and not return any value to the person who originated the data. It’s possible to run greedy descent on that, but you need huge amounts of money. But it’s now possible to get it from people who aren’t thinking very deeply. I I don’t think it’s bad to build systems you don’t understand. But I think this level of detachment from reality is unusual for human history. This episode is supported by Cyber Fund. If you’re building at the frontier of AI, they want to hear from you. Cyber Fund believes the future belongs to AI natives who want to achieve the impossible. And that is why they’re introducing the monastery for AI native founders. It’s an environment of pure focus and rapid execution for founders operating at AI native speed. And they’re offering teams $2 million each to participate. Apply now at cyber.fund. And what do you think about the term AGI, by the way? Uh, AGI to me is just a bit of it’s it’s a PR term. Uh, and it it’s uh some people think it’s it’s fun because you have to have these great aspirations. I think it’s just distortion. I think it confuses young people. And as I will talk about today a little bit, I think that uh one of the things I uh find most alarming about the the so-called thought leaders that uh one will see often on podcasts and uh other venues is the alarmist tone or the exuberant tone. And I think 20 and 25 year olds are watching that and and saying am I going to be exuberant or I’m going to be alarmist? Those are the two choices. And I hope that this conversation we’re about to have is uh one that uh makes it clear to young people that there is other ways to to approach life and in technology. I’ve never actually thought of myself as an AI researcher. I didn’t h read an AI book. The the term was coined in the in the 50s and John McCarthy and others had particular goals in mind for coining it. Um and they had particular methods in mind like logical inference and so on that didn’t really quite pan out. In the meantime uh in the 60s and 70s you know 80s something arose called machine learning the actual methods like decision trees and nearest neighbor and uh logistic regression and hidden markup models were developed in other literatures mostly statistics operations research and so on and that led to industrial success stories. So supply chains and commerce and uh transportation systems all used and still to this day use vast amounts of machine learning. they used gradient-based methods and you know the cloud was developed to handle machine learning workloads at Amazon in fact uh and so that’s the tradition I came up in I was trying to think about systems building uh at scale um that would also serve multiple people the AI buzzword returned I think you know maybe five or so years ago uh because the the data that got to be started to be used was language data and so the box now is not just making predictions about supply chains or commerce or prices or whatever it spits out a human fluent language and people said, “Oh my god, we’ve solved the old AI problem.” In fact, in some ways by if you define the AI problem narrowly like the touring test. Yeah. But there was this ongoing tradition of machine learning and by that time had incorporated people from all different kinds of fields and it was uh really having an impact industry still is. Um but the AI buzzword returned because of loss and now to my view it’s been a distortionary effect on the path of research on how we think about where research should go but also on the path of how do we think about business models and how do we think about where technology is going and AI wasn’t enough they had to create this big hyped up buzzword AGI which uh we will talk a lot about economics you know as a source of of intelligence a social intelligence and when it’s put together with machine learning style intelligence uh you can now talk about at scale, not just numbers of computers and amount of data, but numbers of humans. And that’s critically important to me that uh the role of humans as as producers and consumers in these emerging systems should be should be respected, amplified, and thought about. Professor Michael Jordan, it’s such an honor to have you on MLST, especially given that nature said that you were the most influential computer scientist a little while back. It’s funny because I was trained as a statistician and a cognitive scientist, but I’ll take it. Amazing stuff. Well, um, Michael, you’ve just published a paper called a collectivist economic perspective on on AI. Give us the elevator pitch. I was never an AI person. So, in some ways, it’s easy for me to come in and look at people who are self-professed AI researchers and sort of say, what are you doing? What is your what’s your what’s your point? What’s your goal? I think sadly they often don’t have a very clear goal. It’s it’s that humans are intelligent. Uh humans are a computer. Uh the brain is a computer and um if we mimic that and take aspects of it and and uh parallelize it and uh power make it more powerful, it’ll just do great things and and it kind of stops there. It’s not that there’s a goal in, you know, society that we’re going to try to do this or that. it’ll just solve problems for us and then we’ll be uh happy and it it you know I I got away from Silicon Valley partly because that’s just the way that people talk and I got tired of it. Um and there’s not a lot of intellectual you know let’s call it deeper long-term thought going on. Uh and now became a rat race and a money race and all that. So um so yeah my my perspective um I mean it comes from a long tradition of other people having sort of social science um perspectives on intelligence. We are social animals and a lot of our intelligence comes by the fact that we aggregate we aggregate opinions and thoughts and you know we have cultures and so on that retain them and um moreover the the society provides a context for our intelligence a smart action in one context is not in another context and it’s all very fleeting and contextual in in the moment um and so social science ideas are needed to appreciate what that means when I say social science I include economics so game theoretic the context X is somebody else out there is trying to take advantage of me or maybe to collaborate with me and I don’t really know and so I’ve got to put off feelers and do signals and uh create mechanisms where we can interact effectively and economics studies that in a mathematical way that attracts me because I am a mathematically inclined person. I’m not a critiqueer of AI. I want to make it right and I want to make it better and understand what it means to be intelligent in this world and and safe and interesting and think about long-term issues. And so to me uh you have to do that you know formally or mathematically at some level. It’s not enough just to build things and put them out there. So when I say collectivist I just mean that most of this technology is based on inputs from bill billions of people. So there’s already a collective putting input in and it’s meant to serve billions. So there’s a collective it’s serving. So there’s really a big network that’s kind of latent there. And then economics critical. I don’t want to just sort of you know say words. I want to say I want to write down uh actionable mathematical ideas. This is interesting, isn’t it? Because I think in the 1970s Drafus came up with this idea of the first step fallacy and uh you know so we we create something and it’s related to um you know the Mccord effect as well. We create something so amazing and we just think we’re only one step away from being able to do anything. So these systems they’re incredible, right? They they they produce beautiful text. They can solve problems. they can do programming. And isn’t it weird that they don’t actually help us that much? We thought it was going to revolutionize. It’s not weird at all because uh the the the model there is the old AI model. Let’s just build something intelligent and and it only got upgraded a little bit. It’s going to be a better search engine. That’s fine. I do think the search engine was ma major progress for humanity. But now it became more than search. It was like a secretary sitting on your shoulder uh helping you, whispering things to you. And it’s just a dumb business model. I don’t think many people really will want that. They’ll turn the damn thing off. They they want to think for themselves. They want, you know, maybe at the end of the day a summary or something, but they don’t want this all the time, you know, they’re interacting with this entity uh thing. It’s not a very good business model. Um and in the meantime, we have huge health care systems and transportation systems and finance systems that are all based on data flows among many billions of agents and um you know, are ripe. They already have a lot of machine learning in them and they’re right for thinking in a more economic way. What are the agents and what are they trying to get out of it and what kind of cooperation and competition is latent there that you could you know make improve you know markets arose thousands of years ago and you know we learned about some of the principles but we can improve them. Um and thinking arose you know how you know billions of years ago or whatever. Um, but we’re not perfect and not just in terms of thinking, but we’re also not perfect in terms of narrowly following our own agenda and hurting other people even though we don’t want to. Humans um are wonderful. We we want to prize human life and creativity and you know emotion and love and so on so forth. Uh it’s fundamental but uh humans are also bad or do bad things and uh that’s where technology should be able to aid you. And so you need to think about the system, the OE set of these systems. They’re not really systems. They’re, you know, big statistical boxes uh that do inputs and outputs. That’s not a systems way of thinking. There’s a lower level system, of course, the computer system. But I want to be above that. I want to say what ecosystem does this belong to? Uh who’s it interacting with at what rate and what kind of quality and what kind of values are being created? And and when I say value, I mean often mean money. I want jobs out of this thing. I don’t want just it to answer and do things for us. I want it to create opportunities for for work and and creativity and so on. Rich Sutton is quoted quite a lot um in respect of design versus evolve and um I I watched a wonderful talk by David Deutsch the other day and he was kind of talking about explanations and he said that you know physicists obviously go for these principled low-level explanations but sometimes you get these high level courseings that are just really good and maybe economics is one of those but what do you say to folks from Silicon Valley like Ilia Sutskavar and they’re they’re just talking about human value functions and and they’re saying okay well you’ve got these LLM and we just turn them into multi multi-agent systems and we get all of the economic stuff that you’re talking about for free. Like what would you say to those people? I mean, it’s just not a good way to think about engineering. I mean, if you were a chemical engineer back in the 40s and 50s saying we’re just going to throw a lot of stuff together and make it work. Well, you could do it, but you’d get a lot of explosions and a lot of economically nonviable things. You’d hurt a lot of people. And I think a lot of these people are not thinking about all the people that are being hurt already. You know, Facebook and and so on. You know, it’s it’s it’s damaged a lot of young people. A lot of teenagers are having mental health problems. And this is just uh not something has been talked about by computer scientists at all. And now we’re talking about yet another level of displacement of, you know, jobs may go away, but you know, tough that’s tough. It it’ll create new ones, of course, like always. You know, I just don’t like to talk that way. And you know, so you got to say, well, step back a moment. What is your point? Are you trying to create um uh uh a a a new kind of market where people could come in and have their talents valued and appreciated and where bids could be put out for things that people might need and you know and and collaborations can emerge and uh there could be producer consumer relationships being explored and understood and developed and this could all be a mix of computation and humans. I think eventually we’ll all kind of merge but you know along the way just doing something so disruptive uh with all of these metaphors that’s not good social science or not good mathematics. It’s just metaphors and yes you can build it because the previous generation of people created these amazing things that collect data and we can do gradient descent on it and and ad hoc architectures and yes that works. It’s amazing but let’s not give so much credit to the people that did that. It’s the people 20 30 years ago who did that. So the current generation is just way too you know there’s not much thought going on not much intellectual stuff. Uh it’s just yeah it’s possible to build it. It’s possible to steal the data from wherever you want to because that’s what the internet allowed to happen and not return any value to the person who originated the data. It’s possible to run greedy descent on that but you need huge amounts of money but it’s now possible to get it from people who aren’t thinking very deeply. And so you know I’m I’m see maybe seem more dark than I want to. I mean there’s some lot of good in you know builders but we also every previous era of engineering development electrical engineering chemical mechanical and all had some builders but they had a lot of concepts and they had a lot of thinkers in fact all of those engineering disciplines had something like Maxwell’s equations or Newton’s equations to help them kind of here no it’s just people that are very smart and who can code uh and then have lots of intuitions and and it seems to I don’t ever see anything that feels It’s deeply intellectual to me. It feels like science fiction. Well, I suppose another thing that doesn’t help is that these these systems are like soup. And there’s even a field called mechanistic interpretability that tries to kind of dig into the soup and and it’s almost like they’re they’re searching for UFOs. They’re trying to find these principled circuits that do reasoning or or do whatever the thing is. And um I guess you could say cynically that it’s not like when engineers build a bridge. Well, I’m I’m a little less negative than that. I I don’t think it’s bad to build systems you don’t understand. But then you’ve got to kind of put things around it. And the things that are around are like buzzwords like AI safety. It’s a buzz word. Okay. What you really need I mean the human I you can’t explain to me why you picked this Airbnb over another one or whatever. All the choices you’ve made today are inexplicable to me. They come out of your brain. Um and I don’t need to know all the wise and wherefors of your choices. And uh what I need to know is that you’re somewhat predictable and that if I make certain options available to you, you know, you’re likely to take this one versus this one and therefore I can make my own plans and we can start to interact and so on. So that’s part of economics is the economic style of thinking says I don’t understand all these other entities out there but there’s certain you know rules of thumb that I can use or you know quantitative predictions I can put put in place that allow me to interact and not get hurt and even get value out of it. Um you know so no I don’t think it’s necessary to understand all the details. Now uh the input output behavior you often have to understand better than we can now. For example, if I’m denied a loan at a bank and the bank used this big AI program based on past data, I want to know why. And why doesn’t mean that you look in the internals and show me some circuit. No one’s going to want that. They’re going to want, well, there was like 50, here’s like 50 people that are pretty much like you according to the, you know, embedding we’re using in this big network. And of those 50 people that are like you, some of them got the loan, some of them didn’t. And here, here, let me just show you what those people are like. You start see, oh, I see that they differ from me in this way. that’s actionable to me. I could now change things. So, you have to build systems around this predictive system. That’s a nearest neighbor system, for example. And that system will supply what people might consider more like an explanation. Um, and so it’s not just trying to go in the internals of something. Again, chemical engineering is, you know, there’s there’s there’s certainly thermodynamics and lots of things are understood, but lots of phenomena were not understood for a long long time. you mix up a bunch of stuff and you know certain waves are created and certain things happen and you exploit that and go and move on. Um but you understand something about input about behavior and uh and constraints and and so on. Um I think the current generation of neural nets will continue to you know they they have very nice scaling behavior. They’ll continue to be there. Um but they really have to be thought of as part of a bigger ecosystem. And then you kind of ask well what can the neural net do in this context and what’s it missing and uh what if I have multiple of them and uh you know how do how do they engage with each other and with us and how what what is needed what’s what transparency is needed for the overall interaction to be an effective one whether or not I understand all the details or not for some reason and correct me if I’m wrong I I have an intuition that behaviorism is bad that just by not having any mechanistic understanding and only looking at the outputs there’s the famous example isn’t there of of of the um the hen didn’t know his neck was going to be broken. And one example of this actually is um alpha fold. So I interviewed John Jumper last week at Google and you did some analysis on those 200 million predicted proteins and and and you found they were very good but there was something missing but you could robustify them. You could robustify them. That’s correct. And I think that’s a good example. So you know I’m a big admire of AlphaFold. I don’t think it’s like an LLM. I think it’s uh you know targeted. It was for a particular set of problems and it does it very well. Uh the issue that we found empirically uh was that when you ask certain kinds of questions um you know in particular we did one where we were looking whether quantum fluctuations in a protein um were associated with phosphorilation meaning the the protein was active or not in the cell and you might think that these fluctuations which lead to strands hanging off or kind of like you know bad proteins evolution wouldn’t use them but it turned out that a lot of them seem to be phosphorolated mean they’re reactive in the cell. Hm. that suggests a hypothesis test. Is there an association between yes, no phosphorated and yes, no quantum fluctuation. So that’s a little 2 by two table and you do a statistical test on that. And uh the problem is that if you just use known protein data uh that’s been crystal you know there’s a crystal structure known um you don’t have enough data to to test that hypothesis with high power and so you can’t reject the null hypothesis that there’s no association even though there looks like there is. If on the other hand you use 200 million you know proteins out of alpha fold you can test hypothesis with high power and you reject the null hypothesis. But what we found is that the um confidence interval on that statistic of that 2x two table was extremely narrow and way far from the truth the true value of the uh the the gold standard value. And we found this in domain after domain you know. So why is that? Well, what’s happening there is that there’s probably not many um examples in the training set of proteins with quantum fluctuation because it’s not been that studied in the past and it’s hard to crystallize and so not many examples means it’s quite possible alpha won’t give out a great answer but it won’t tell you that it doesn’t give you out air bars and it doesn’t spec but specifically on the question you’re asking that’s where I want the air bars and it didn’t know about that question when it was you know built and designed Okay. All right. So now I have a good statistical question. What if I add a little bit of ground truth data to the 200 million? Can I shift the error bar so it stays somewhat narrow? So I have high power, but it covers the truth. And the answer is yeah, there’s a methodology. We’ve developed something called prediction powered inference that does exactly that. And so it’ll cover the truth just like in a classical statistical setting, but it’s using this rather highly biased architecture. And it’s now it’s not biased overall. In fact, its accuracy is high overall. But for the question I’m asking, it might be very biased. And that’s going to happen a lot in science because scientists are rarely interested in just studying the past over again. They’re interested in brand new things on the edge of knowledge. And that’s where specifically these foundation models will be most poor and most highly biased. So there needs to be around any foundation model the ability to maybe collect a bit of ground truth data to merge it in with some procedure like this and then to give out a more trustable answer. That’s all not science fiction. that’s what can be done and what really needs to be done and I’m sure the AlphaFold people are on board with that that they would not find that weird or surprising. Um, but a lot of other people out there talk about bias and all that and they either don’t worry about it. I say it’ll go away if we have enough data or they just critique the architectures and critique the outputs but they have no scientific I you know method in in mind that’ll help us go forward. So that’s kind of the state we’re in. I challenged John a little bit about the extent to which AlphaFold understands and he was basically allergic to the word understands. We are not trying to tell you everything. We are not a model of the entire cell. These machines let us predict. They let us control. We have to derive our own understanding at this moment. Right? We can experiment now on the artifact. We can look at the 200 million predicted structures. not just the 200,000 experimental structures in order to help us understand, but it doesn’t do the act of understanding for us. It does the act of predict and maybe control. Why why should AlphaFold understand? Well, what would it mean to I mean, he was he was sketching it out to me. He kind of said that this this is a weird alien artifact and it’s not like it’s kind of created. It’s refined. There’s this recycle pathway. You can put the thing through multiple times. you can kind of corrupt it halfway through and the network is just iteratively kind of you know it solves the complex bit first and then it’s refining refining refining and like could we interpret that as an understanding process I don’t think we need to see I think this anthropomorphizing of intelligence and understanding all that is not necessary not appropriate and is is a distraction for many many problems why say it understands you know some of my heritage comes from seeing in in real life in in industrial settings machine learning algorithms being rolled out 20 30 years ago so When I first went to the west coast, I visited Amazon in around

2000. They were using huge amounts of data to do supply chain modeling using the neural networks of the day. It was random forests and it was really working. They could make really fantastic predictions of you know whether certain ships would be delayed in the Indian Ocean or whatever and so certain parts wouldn’t arrive in time and the overall supply chain takes billions of products and sends it to 100 millions of people per day. And so you cannot there’s no way that any human can understand what’s happening in that big big u box. Um but it’s not necessary. Uh and in fact you can ask does that overall system understand you know transport and logistics and the answer is who cares. It’s it’s it does a a very important optimization and and prediction process that allows an engineering system to be built around it. It brings down uncertainty. It makes you possible to do kind of stockpiling and you know planning and that’s what you ask for. You don’t care whether it’s uh has to have a word like understand it or intelligence applied to it. That’s for the media. The media that’s that’s kind of my problem with a lot of these people rolling out AGI and and AI terminology. The media laps it up and they know that even though we don’t have a clue what understanding intelligence mean and we we in our own research realize we don’t care or need it. We want to build good systems.

Yeah. It’s interesting because I agree that we live in this complex adaptive irreducible system. we can’t essententralize it and uh folks like France or even David Krakow they talk about intelligence as the you know adaptation synthesis of course grain representations but what if there is a bit of a step so let’s not anthropomor anthropomorphize it let’s say that understanding is about like not not the end point it’s about the path which led us there and we know that in the real world we’re a collective intelligence and there’s the blind men and the elephant and we all take our own paths and lives and we we have different perspectives on the same hole so what if like a better form of understanding is just being able to reconstruct the thing from your perspective using building blocks rather than trying to essentialize it. You know that that’s all sounds great. It’s just not the language that those of us do research would use. I mean we would think in those terms a little bit of course but um we would try to turn it into some kind of an equilibrium or optimization problem and here’s the information that’s available and here’s the data and here’s the power and the you know the the error rates and we try to put a little bit of structure around it of that form. Um and and you know there’s always this creative moment like I remember when in high jumping I used to be high jumping interested in high jumping when I was a kid and you know you would go up to the bar and you jump over it in various ways and there the barrel roll rolling ac you know um was the was the technique the the Olympians were using and then there’s this guy came Dick Fosbury came along and he says no if I go backwards I can do better and no one had thought about doing that as soon as he did it everybody did it and that’s and and the and you know the went up like by a half a meter or something I don’t know. Um and and so um what process led to that? Was it an understanding process? You know, it was just a little bit let’s try something different mixed in with uh the ability to try it out and uh to do tests. So a huge amount of industrial planning is uh try it out and see what works. Um those are called AB tests and those are done all of the time and I’ve got nothing against that. It’s not based on understanding, but it’s led to optimized systems that can do things that, you know, people hadn’t thought about before. So, a blend of that with understanding, but just understanding, you know, I I was a cognitive scientist and I, you know, and interested in neuroscience. I’m one should be interested in those things. They’re fascinating, but they aren’t they aren’t the leading edge of thinking how to build systems that, you know, work in the world and they’re not the leading edge of trying to build even the next generation systems. If you put a lot of people keep saying well we got to put logic back in or symbols because that came from the pre that came from our previous kind of view of what humans are doing and probably humans are capable of doing some logical reasoning and probably have some symbols whether they’re kind of built in some complicated network or they’re reified somehow. I don’t know. Um my intuition is as good as yours. Um but really the goal is to you know in a I I tend to be an engineer at heart you know a mathematicallyincclined engineer. I want to say what are you trying to achieve? Are you trying to displace teachers? Are you trying to make doctors better? You what are you trying to do and what what would be the abstractions and the and the the points of entry into that problem and then how can you kind of pull back from that and do it in some general way that’s that’s elegant will inspire others. It’s so interesting seeing different scientists, you know, from a multi-disiplinary perspective attack this problem. Um, physicists, for example, they they work very very low level and they talk about, you know, the the dynamics of particle systems and whatnot. And um what I’m really fascinated in, I mean, you come at it from an economics perspective which is traditionally um dominated by this agential lens and you talk about equilibria and incentives and and so on. How how does that come into it? How would you take a very complex system and almost kind of um decompose it into this new frame of thinking? It’s not been done really enough for me to have you know tons and tons of of great examples but um you know we’ve been looking at kind of modestly scaled examples where there’s a um so for example we looked at a little bit of drug discovery and kind of the the regulation you know so I’m a pharmaceutical company I test out all kinds of proteins and I throw them in animals and maybe a few humans to sort of see what’s working and I have some understanding quote unquote and other I know somebody the evolutionary biology behind it and so on and that guides me but at some point someone’s got to really test this out in the real world and decide regulatory agencies got to come in and say yeah that goes to market or it doesn’t. Okay. So now you got a kind of tangled web of scientists and pharmaceutical companies not just one but many many of them and proteins and um uh and now you’ve got to think about how that system is behaving. So the hopefully the regulatory agency is trying to overall over the entire system have the number of false positives be low and false negatives be low. That’s what the goal is of the problem. So it’s a statistical problem. Oh, but wait a classical statistical problem. You would just go gather IID data, independent identically distributed data from some source here. No, the data is coming from the self-interested pharmaceutical companies. What’s their motivation? Money and whatever. Maybe to help. They want to help people and money. Um, and all that’s kind of hidden from you as a regulatory agency. Okay. So now the economic mindset kind of comes into play. He says well it’s hidden from me but there it’s not arbitrary you know I can kind of probe in various ways and so that becomes very economic so you know economic economists think about how you set prices you know so if I got a lot of people coming on my airline uh there’s a thousand people that just arrived who want to go from here to London um every one of them has a different price point and that price point will shift in the moment how how eager are they to get it’s not just because they have a lot of money it’s because they have needs and I don’t know what those are so what I do is I set various services and various prices that kind of bracket the possibilities so that overall it’s likely I’ll make enough money and they will everybody will be kind of happy and the service will go forward in life. That’s what so it’s a blend of knowing a few things and u admitting that you don’t know other things but putting it in a system that actually can work with that kind of uh mix of asymmetries and and and incentives. So, you know, the incentives there are that you, you know, uh there’s a certain service and price. If you pick that one, you’re likely to be able to get on the airplane. You’re likely to get, you know, have the the goodies you need or whatever. And, uh, that then doesn’t make you do something. It incentivizes you. And in the pharmaceutical world, if I could get them to be incentivized to mostly send in drugs they’ve done some testing on or they have some belief it’s a pretty good one and not just throw arbitrary ones at me, then maybe the overall system will actually have the air rate you want it to. Okay? Because if you don’t do that, um, then if it’s a drug that’ll make a ton of, you know, a billion people will use, uh, then you’re going to make money whether it really works or not. Okay? So, you need it to get to market. How do you get it to market? Well, you just throw it at the regulatory agency and there maybe is a false positive. They just, you know, got a false positive and they put it on the market. You make a ton of money. And if there’s enough of that, the incentives are all wrong and the overall system will not control type one and type two errors. Uh those are examples we’ve actually worked on. But I just hope you can appreciate when all this stuff starts to really roll out in society, it’s not going to be there’s a few big LLMs and everyone consults them like a search engine. That’s just not the model. It’s going to be there’s local data like I told you about with prediction power inference. Everyone’s got to vet what what’s coming at them. Uh there’s going to also be local data because I collected it with some expense and I want to just give it away. Okay, thank god finally Anthropic is paying people for for money. You that has got to be the future. So I’m going to have some competitive value in my data and not just give it out. All right. And uh so now if you start interacting with lots and lots of people that want to get some value out of the interactions, you have to talk about the incentives. What’s the incentive for them to send the data? But not only send the data, send correct data. Send truthful data. don’t just add our you know not be adversarial. Um and so I cannot imagine uh you know a fully full-fledged version of all this rolling out in society and all of our decision-m throughout our lives uh without a deeply microeconomic perspective accompanying the gradient descent on data. You you spoke about this three layer model. So um there was an example where you know you might have um you know consumers and they might have their data and you you’ve got Google and then you know Google is using the data the consumers are getting a service and then Google might sell the data over here. That’s kind of like a traditional model. Let’s start with that. Okay. So those are really kind of like bore atom kind of things. We’re being scientists there. We’re trying to say what’s a minimal model that exhibits some of the behavior that we want to study here. So let’s think about a data market because data is not just now something you analyze to build a big LM. It’s also something you’d sell and buy and has value and also there’s privacy concerns about data. Okay. So let’s put a little minimal model together where we could study that. All right. And so one we’ve done is called we call it a three layer data market. Um and it has uh it is it exists in the real world. This is a abstraction but it’s a it’s a real thing. You’ve got a a user or multiple users coming into some platforms. the platforms provide a service like you know imagine you know payment service and as I use that they get data from me they learn about what kind of purchase I’ve made and so on and they use that data to make their service better that’s a good little nice little loop there right uh the problem is that rarely do they make enough money off of that service they you know take a small cut u that the merchants don’t like to give them um so they have to do other things to to to stay in business so typically now for a long time now um probably 20 years they’ve been selling their data to third party data buyers and these are not evil people um just trying to you know ruin people’s privacy. They’re they’re trying to do market research learning what what would work and uh what what are people really doing. This is behavioral studies and so that there is value to them. They pay for it. All right. So, you know, Google doesn’t need this because they created this artificial advertising market which we could talk more about that kind of superpowered all this nonsense. Um but uh other companies like Mastercard would do have to have to sell their data. So, so now it’s a three layer thing and and as soon as that third layer was uh introduced, the um the equilibrium has to shift because the um the user who’s sending their data in just lost something. They lost a little bit of privacy. Some third party that I don’t know anything about is getting data about me. Um and I can’t just accept that. Um you know, but I can’t walk away, you know. So, there’s a stress on the system now. So um in an effective economic system what would happen is that the uh you wouldn’t just wait for the regulator to come in the government say you know no this can’t be done what you would do is that the platforms would say well we’ll offer you uh a tunable level of differential privacy for some cost or we’ll just say that this our company I’m Google I’ll offer you level you know.3 and some other company says well I’ll offer you level 7. Okay. So, the user looks at that and says, “Ah, 7. That’s that’s better.” Um, I I really care about my privacy, so I’ll go there. That company then will get start to get more data and their service will get even better. And oop, you got a little nice little feedback loop there. But now the data buyers will look at the data from that person. At 7 means more noise has been added to the data. It’s less valuable to the data buyer. Data buyer will say, “Oh, I’ll I’ll spend less. I’ll give you less money for that. I’ll give more money to Google.” And so now you can see there’s conflicting um tendencies here. The the incentives are aligned but they’re not uh you’re not optimal for everybody. Um and so now the mathematics is not just an optimization problem. The mathematics is an equilibrium problem. But it’s an equilibrium problem that involves statistical uh assertions data and how much you can predict with this data and so on that. So you quantify that with error bars and and statistical predictions. So you put that all together in a big mathematical system and you can find the equilibria as a function of various system parameters. So for example, is there a minimal level of privacy the regulators could require or not or you know is there some heterogeneous privacy budget you know etc etc. You can put in various uh those and now you you do a little plot of how the equilibria moved and the equilibria have overall utilities for all the three players summed up. That’s the social welfare. You can ask how high is the social welfare or that equilibrium versus this one versus this one. And another regular could look at that say well I prefer this one because it’s overall higher social welfare and laws could be made at that level. Okay. Okay. So, even though this is a toy little, you know, little toy model, uh it has the ingredients that I’m very interested in. Predictive models, data markets, but uh money incentives and uh a real system that really is already kind of working, but people aren’t thinking about it very well just like in the drug discovery domain. But if you take an economics point of view, uh you can make a lot, you can make the system better, right? Cool. So modeling it as a dynamical system which we can simulate and then we get these modes of no we don’t have to simulate it in that case it’s you can actually write equations and calculate equilibria it’s it’s a stackleberg game and you can actually find the equilibria but you know in other cases you would simulate but the point is a lot of my machine learning colleagues don’t know much about fixpoint algorithms and um finding equilibria and how they shift as you shift various parameters and all that that’s economics stuff um machine learning people are really good at optimization but this is not an optimization problem. It It’s a and there’s all these algorithms in in um other branches of mathematics that find parto frontiers and and do it statistically and and uh do it as a function of size of of various markets and size of of populations and all that. And it’s kind of amazing in this era that the two have almost never met. The economists never had a lot of data to inform their design of their market. So they just wrote down a bunch of equations and made rational assumptions and all that and then found equilibria mathematically or otherwise. And the machine learning people never thought about the equilibria. They just had a lot of data and they used it to do the obvious thing predict the next word in a string of words. Uh but the the future is got to be that those branches come together. The economics equilibria perspective is critical but the oh it’s got to be adaptive perspective is critical. And also you alluded to earlier some of the machine learning uh Silicon Valley types just saying well we’ve got all this data therefore all the behavioral stuff’s already built in. Um and and that’s that’s too naive obviously. Uh but it is a useful point of view um in a certain sense. The economists uh do make rational assumptions they shouldn’t have to make and if you put in data instead of that assumption you’ll probably do better. You’ll have some of the behavioral economics already built in. But if you do it outside of any economic thinking whatsoever you’ll just make a mess of things. And of course that’s what Silicon Valley seems to be pretty good at. What what what is the difference between like data and the kind of knowledge that you’re talking about? Um social knowledge is very ephemeral and it’s very in the moment. Um you I walk down the streets of Copenhagen here and there’s all kinds of little markets out there and what’s available and at what price and you know what I might like and all that. It’s all super ephemeral and that that’s kind of I think a better way to think about all this. You can’t just gather enough data to know that that person walking down the street there, they’re going to come buy this product. It just and all of our decisions and our choices uh you know cannot be uh that that you have enough data to cover all that you know everything about what’s going to happen even in the next you know 10 seconds. Um, so you have to be a little more humble about, you know, that I I have a lot of ignorance, but that doesn’t mean I can’t build a safe system like a market of some kind that could people could come in and they can not get cheated and they can get value and and it can evolve over time. It can shift in ways and I I don’t have to be I’m not the the the the god figure at the top, you know, designing the human value function or whatever to put it in so that it responds particularly well to what humans really want in my god vision view of the world. Uh, no, it’s got to be a system that permits u bottomup preferences to be expressed in the way the human wants to in the moment and that’s not built in. The system respects those things and wants to maybe learn more about them and then use them in the moment and then maybe keep some of it. Maybe it’s all ephemeral and goes away. But there seems to be a huge naive day about what data even if you have you know whatever exabytes or whatever of data you’re going to miss all the details that are probably the main thing that matters for a particular kinds of class of decisions even alpha fold based on huge amounts of data doesn’t do for well on certain queries and yes they’ll patch those and it’ll do w better and better but it’ll always the the the new questions that people will ask will always be on the edge of knowledge or often be on the edge of knowledge and people aren’t thinking about that they’re thinking Well, I just got to replace the teacher because a teacher is working not in the edge of knowledge. They’re working back in the all the stuff that’s already known. That’s fine. Uh you can aid teachers, but good teachers also kind of know how to migrate to the edge of knowledge. Yeah. And when we do abstraction and idealization, it’s always a little bit lossy. And I’m I’m fascinated by this observation that market, you said markets were around before capitalism. So, it’s this bottomup thing. It kind of it’s a this is not capitalism. That’s one meth one methodology for make markets work, but it’s not the only one. Exactly. So it’s a it’s a it’s a natural phenomenon the the you know the emergence of something we might call markets and it’s constructive and it’s divergent and diverse and then you’re saying somewhere the rubber can meet the road so we can create abstractions we can do some kind of modeling and we try and do it in such a way that we don’t it’s not too lossy absolutely human culture creates abstractions individual humans create abstractions too that work for them and and when those abstractions are kind of useful enough and they can communicate and kind of get promoted into the culture and that flows up and down all the time. Uh and indeed that’s something that systems could perhaps help with. I’m not going to just trust systems to take on that burden but that could it could be helpful and so indeed uh it’s not just the individual cognitive entity that creates abstractions and we should just reify that. Yes, it comes but cultures create abstractions and you can study the micro economy of that or whatever or not. you can just sort of say those abstractions came are part of the culture and they’re useful. They’ve stayed around and and and they’re useful and and going forward it’s not that we’re going to just keep the old ones but we’re going to build systems that allow new ones to emerge and that is not the godfig figuring it all out and putting them in there. Um I I you know so Silicon Valley says we just got it we got so much data and we’ll have so much that we can do all of it top down and they’re forgetting somehow that first of all the data came bottom up. the data was all contextual and the data was supplied by people and so on, but they’re also forgetting that it all has got to continue to um be at a micro level that is kind of going to be beyond their ability to sense and we’re not going to want them in so much in our lives. You know, the after the search engine, which I thought was a fantastic piece of technology, allowed access and all that sort of then a lot of it was very prime. We’re going to put glasses on you. we’re going to put, you know, things around cameras in your home and all that and we’re going to we’re going to know all the details of your life and we’re going to, you know, make your life better somehow. And that just that equation did not calculate for me. Yeah. So this idea that that culture is the abstraction hard drive in the sky and but culture is very adaptable. So we can delete strategy. I mean knowledge decays very quickly and organizations maintain knowledge and really good bits of knowledge stay around for a long time and you know down at the bottom up we’re creating new bits of knowledge. So how does that whole ecosystem work? How how do we kind of designate good things that stick around and how do we find new bits? We don’t. You and me the answer is you and me don’t but um that’s something that uh good intellectuals do. Um that’s what you know economics like there’s a whole field of behavioral organization or you know how does how do organizations effectively emerge and those that’s really interesting. It’s not everything but you know there’s a lot known there and some of it’s mathematical and some of it’s not some of it’s best practices. Uh but those are the kind of uh ways that these e these AI ecosystems should be talked about and not just in terms of neuroscience and you know metaphors of the neurons and and physics metaphors and all this stuff that it was part of my heritage too but it just felt felt so lacking when we actually see these things the rubber hitting the road as as you say. uh and so yes behavioral organization how people are organized into things that promote you know not only you know good revenue for companies but also promote democracy and and so on so there are people that talk about all these things and I just don’t think they seem to have much presence in Silicon Valley um and maybe that’s for good you know Silicon Valley let them go you know they’ll just burn a lot of money and you know cause some headaches and and they’ll also create things like search engines and um and I think a lot of the companies are also really focused on creating value. I do think Amazon is different from Meta. Amazon has got a business model bring packages to doors and behind that they create some technology to support that and it’s mostly do the good in my view. Um you have to worry about labor markets and so on so but those are all things good things to worry about but just um creating uh computational artifacts that make predictions and that if you were to wear goggles you would be able to you know live in their world. Um it’s not a business model. It’s a it’s a science fiction dream that it may or not be may not be helpful for humanity. Well, can we explore that because you gave the example of um Spotify. So, you know, we were talking about this three layer thing before and and now we’ve got this weird incentive structure where Spotify are actually incentivized to generate the songs with with AI, right? Yeah, they are. All right. And I I’m a you know I have a a project. I’m a scientific adviser to something called United Masters which is an alternative uh which has uh musicians keep their their their work and um and United Masters uh connects them to brands and to other kind of opportunities so that they are kind of more like a real artist. Um not just like that they their their song got streamed and they got a little bit of money. uh because Spotify indeed is not it’s close to perhaps a monopoly but uh it’s not incentivized to pay pay there’s not a pricing there’s monopoly prices if you will um and so one would hope that somehow the market will fix that that enough young artists will say I’m getting screwed here I’m not making any money and another service will emerge um but we are in an era where some of these services uh do become monopolies pretty quick and that’s I leave to my economist friends to think that through and to look back at historical examples and think about, you know, is regulation needed or is um or is there other market making mechanisms that’ll make this more healthy for human beings? Um I’m not against Spotify. It’s a but you know, it should be part of an ecosystem that actually rewards the artist more. Right now, an artist is getting paid very very little and I think I’m not don’t believe the prices are being set under competitive mechanisms. But I I think there’s this broader, you know, macroeconomic view of what are these systems doing and what are they what’s their role in society going to be? And with the search engine, I was many of us were puzzled about how they would make money. It just didn’t seem like, you know, there was a money-m and then the whole advertising thing was a bit of a surprise, at least to me, that it would become so huge. Of course, the underlying thing is that people expect things for free. All right? And so Google couldn’t kind of make payments, but I think they made a mistake at some point. I think like with YouTube, when they acquired YouTube, YouTube is more than just pointing to people to a website. you know, YouTube is uh incentivizing creators to create things that people will watch. At that point, I think a socially responsible Google to critique them a little bit would have said, “Oh, we’ve created a market here. We’ve created a producer consumer relationship.” We’ve got to make that market a little bit more valid and we could actually have that when someone’s watching things, they can have a some sort of a economic connection to the person who made it and there can be incentives flowing that this person is now incentivized to make more because here’s my audience connected directly. instead it was all going through Google and then Google was putting advertisers next to make a ton of money for themselves and then there’s a modest incentive to give back a little bit of money. That to me was a huge mistake and then Facebook made it even worse. So you’ve buted up against folks like um Jeffrey Hinton and Stuart Russell at Berkeley and um these guys are painting a picture that this technology is recursively self-improving that it is aential that it’s not a cultural technology it’s a thing in of itself and this seems a little bit science fiction on the first read very science fiction what do you think so I think it’s science fiction and I think science fiction is important for society but it’s also at the level it’s being promoted and and and those kind of voices, it’s really hurting 25 and 20 year olds. You know, these these young folks of whom there are huge numbers are excited about technology and they want to build things that help their family and help their country. Actually, more of their family than their country, honestly. And they they they see real opportunities in doing that and they’re kind of being told by the leaders, well, we had our fun. We developed a bunch of algorithms. We we did it and we were just interested in the pure, you know, understand intelligence. even though they didn’t understand intelligence, they built, you know, gradient descent algorithms. Um, and now you guys, you can’t do this because it’s dangerous. It’s going to it’s going to wipe out humanity with a with a high probability or it’s super intelligent arrive soon, so there’s nothing left to do. That’s in your lifetime. That is so demoralizing. So demoralizing. And that thing I think that bothers me the most. I mean, the second part that bothers me is there’s no economic thinking going on there. It’s zero. It’s really about uh cognitive science mentality or neuroscience. We figured out how the brain works. It’s gradient descent with a lot of distributed neurons and the the fact that these LMS are working so well shows that we figured it out. It wouldn’t work so well otherwise. Well, I think that’s dubious. We don’t the brain is way beyond I mean you ask a neuroscience if this has anything to do with the brain basically they’ll say no. It’s a nice metaphor. It’s cartoon. Uh does gradient descent work at massive scale? Yeah, more than we would have ever imagined. Um but is it showing its uh weaknesses? Yeah. Can it be fixed certain areas? Yeah. You know, you build certain verticals, they’ll do good things and uh it’ll make mathematicians go faster, but it won’t put them out of business and so on. Um it’s having a big effect on society. I worry worry more about labor and capital relationships than I worry about it deciding to take over. Um so the rest of it that that to me is more on the ground sort of you know how does uh a young the next generation take technology and work with it and I don’t think that voices like that are actually helping that tech that that that uh generation to actually perceive what they should work on and why. Um super intelligence versus extinction. Those are your two options. Um and god damn it those aren’t the only two options. there’s a huge number of very positive things that can be done at human scale. Um and let’s hope that enough of the young mentalities kind of get behind that. Um um but they don’t have enough examples of people out there who made money by making did Sam make life better, you know, not clear. And and and uh you know, I think in previous generations there was a little bit more, you know, here’s people that are out there making things that uh vaccines or whatever. Oh, I want to be like that. And right now, not so good. I don’t know if I could um get you to be a psychologist for a minute and try and understand why these I don’t know whether it’s the search for purpose, but have you noticed as well that some folks think that there’s going to be a utopian future and when you when you speak with them, they there’s quite a similar DNA. So they they also think that it’s recursively self-improving, it’s going to be super intelligence and so on. If if if I was to press you to say why do you I can understand right these things are so clever but why is it why do they believe it I mean they’re clever in the way in a recognizable way at some level they’re not they’re taking all this human cleverness and packaging it in in a new way and um again I think it’ll kind of always be missing a little bit of the point because it’s not in the moment it’s not the ephemeral stuff um um but that doesn’t mean it can’t be even more clever and I kind of think that that’s Okay. I think that the for me the goal here is not to build a super intelligence and have it dictate or tell or anything like that. It never was. And I’m kind of shocked that some people seem to think that was always the goal. It to me it just never was. Um rather again I think I said this in the very beginning. Humans are wonderful. Um you know I’d hate to have robots taking over from us and I don’t think that’s going to happen. Um, there’s just too much good about human nature and about what humans are, you know, able to produce that are shockingly beautiful um and creative and inspiring. Um, we need to support all that. Now, the the issue though is that the flip side is that um we are not we’re far from perfect. You know, people really hurt a lot of people and they they are being empowered to do yet more of it. And uh we are very narrow-minded. We also don’t understand like you know people hurt other people often because they don’t understand their motivations. They got a misunderstanding. How many wars are created because someone didn’t understand the intentions of the other side and they said well let’s just you know proactively let’s just bomb them. That that’s just all the time. That’s how humans act and think. What’s missing there is an appreciation of uncertainty and information signaling and sort of eventually game theory arose to help people think it through a little bit but it’s still extremely rough. And if you look at our political system, you know, an an aged, you know, charlatan leading a country, um, you know, this is our optimized human system for making decisions at, you know, the of the highest kind. There’s so much room for improvement of the human being. and democracy’s got to be the way. But democracies right now are a few aged people sitting in various rotundas in in various capitals, you know, not knowing what they’re talking about mostly. Uh we have a very broken human system in many many domains. We have a few that are m I think the universities are pretty good and I think a lot of companies are pretty good and a lot of human associations of various kind various skills are pretty good but we have so many broken ones. And so to me that’s what AI is about. AI is about uh helping the things that were too hard for humans and aiding the information flow so the humans could actually make the good decision in the moment that most of them really wanted to make and and and and and now I’m making the bad decision that they were afraid they had to make because they didn’t know enough. So there’s so much to me opportunity if you think about it at that level that’s what AI is about to me. AI is not about this replace the human with the computer the recursive self-improvement stuff. I mean it just feels like a a metaphor. You know, we work with recursive algorithms. We work with improving algorithms. I don’t see that getting out of control like you know a virus that somehow you know we’re going to work with these systems and we’re going to uh like I say hopefully mostly focus on getting right some of the things that evolution didn’t quite get right for the human being especially at scale of 7 billion. Evolution perhaps didn’t prepare for that. Um and focusing on that to me is what AI can be about. So I’m positive. I’m bullish about AI in that sense. I I and I’m I’m appalled by the dialogue has become between the people that have all the money and want to just build something for build it sake and the people that are just anti-intellectually saying it’s terrible. It’s going to it’s going to destroy all of humanity. That’s the dialogue in in in the public eye right now. And that’s just that’s I find that so harmful. And um and it it does bother me that people that worked on it for all these years think that we’ve reached the end. That somehow the gradient descent is like the brain and therefore you could take multiple brains and you could fuse them together and oh my god it’s going to just do uncalculable things. That’s just such science fiction. It’s it’s whether you know even true or not it’s worth not even thinking about it’s what what’s the path? What how do we engage younger people to do things that are actually positive and what mechanisms are you going to talk about? what kind of education are you going to talk about? What what goals uh are you going to set? And the thought leaders are not talking any of that kind of language. And you know, I think it’s unusual for human history. The thought leaders are are are heading off in these two directions. It’s also complex because there are very real security and safety risks of having any autonomous software, you know, just doing things without direct human supervision. So we we should say well, yes and no. Think about airplanes. You know, that’s the classic example, but there’s very very few uh airplane crashes at massive scale these days. There used to be a lot when I was a kid. And it’s because of the autopilots. Yeah. And mostly now planes planes are flown by autopilots and um and the human can come in as need be. But it’s it’s because of that. So there’s this blend of automation uh with human is actually the most effective way to go is again it’s improving. Humans didn’t evolve to be flying this big thing up in the air and uh so you can improve upon human ability there. You put the two together, you can do something that’s helpful for everybody. I suppose in that case it’s quite a well specified problem. So we want to go from A to B and here are the parameters. Yes. to know. I mean, you have multiple planes in the air. You have clouds. You have, you know, changing weather patterns. You’ve got uh some person who would did something stupid. You know, it’s easier cuz yeah, up in the air there’s a lot of room. In 3D, there’s a lot more room than in 2D. But in in 2D, you got all these cars floating flying around. You got tens of thousands of people dying each year in each country. Um it’s a mess at some level, even though it’s very important and effective for many of us. So, we do it. Um, but a hybrid system that had a lot of autonomy with some human and and so on. But you got to think about at the system level, just putting a super intelligence behind the wheel of a car. Dumb dumb way to think about technology. Is there any hope? I mean, um, I don’t know what you think would be the thing that would make these folks update. I think that Ilia and others have done some great things. I mean, they built some systems that all of us are um, not only using, but kind of it’s changing our thinking and all. And I think that’s kind of what I get out of what they’re saying is that I’m a builder. I’m not a you know you think I’m a guru and a thinker and maybe maybe I think I am too but maybe I’m really better as a builder and I can build things and with uh the resources that are now available to you and again it’s not just the money it’s uh it’s the whole internet and the whole you know all all the things that previous generations of people did that one thing that bothers me a lot about these people not the Elon Musks or the Sam Almans they’re just coming in taking the cream off the top you know from all this effort that people put in and a lot of these people are you know rightly not just they wanted the credit. They just are annoyed that this is the direction it’s that these people are now taking it uh without the appreciation of what why were these people building these things not for you but uh I had other goals in mind. Um so you know I think that um yes these are some builders and um uh there’s some very you know pressy builders but um it it is I don’t and I think there’s this system that we call it Silicon Valley whatever that these people live in and where think the more outrageous the more far-flung the more physics biology inflicted neuroscience inflected that your language is the more you sound like a guru um and and people enjoy that uh that posture and that activity. And uh it creates great amount of money. They don’t care about the wealth perhaps, but it creates them allows them to yet be more prominent because they can now have another company that tries some other crazy thing. And um and so if it doesn’t work, that’s a sign of you had a great idea. Um so it’s a it’s a I wouldn’t want to be in that world. And I am trying to becoming a bit of a historian. I mentioned chemical engineering, electrical engineering, but you know, you look back at the history there there was some glimmers of some of these kind of things. But I think this level of detachment from reality is unusual for human history. This level of my crazy science fiction 25-year-old dreams are all that’s what I’m going to pursue for the rest of my life. Whatever with, you know, come hell or high water. Um, and then at some point I’ll flip because I realize, oops, I didn’t really have a great goal in mind at all. And what have I got here? Oh, I’ve just spent a lot of money and I got this thing and I don’t really know what to do with it and I’m worried about it. You know that to me is a sign of a certain level of immaturity. Frankly, circling back to, you know, you you were talking about this statistical contract theory, which is when, you know, we we we have uh things with an information asymmetry and we model um incentives. Um a lot of folks in the audience would have heard of game theory, right? You know, um what’s the difference? Oh, well, game theory is a discip a mathematical discipline. You know, it started with van in the 20s and uh it’s got many many branches to it. Um and it’s a mathematical way of thinking really. Um and one way I like to think about it is that um it’s like f equals ma. It’s a set of um it’ll make predictions. Okay. So if I write down a game just like I wrote down fals ma in some coordinate system I can now predict what’ll happen. And in the case of fals ma I integrate a differential equation. In the case of game theory, I write down the game and I calculate the Nash equilibria or the correlated equilibria or some other equilibrium concept and I say here’s what’ll happen in nature because my little mathematical model has the kind of appropriate captures the appropriate ingredients and for f= ma yeah the the thing follows a parabolic you know curve it means the theory is right and then Einstein says it’s not quite right and he makes a better one and in game theory same thing you look at okay do those those equilibria actually characterize how systems and organizations and people behave. Sometimes yes, sometimes no. But those aren’t that’s not the end all. So there’s all kinds of other equilibria, Stackleberg equilibria and sequential equilibria and and various kinds of figures of merit, you know, various social welfare constructs, various regret constructs and all sorts of things. It’s a whole huge field of its own. Um, and let’s think about it eventually kind of being as big as physics because it’s all about strategic interactions and so on and you know not not molecular interactions but but now you can also ask the the the inverse question in in physics the inverse question would be I want to build a bridge. So my goal is not just to see if something follows a parabolic path or something. I want that bridge to stand up. So I invert F equals MA. Okay. I go from the goal back to the design that would ensure that that thing stood up. All right? And so most engineering fields are inverse problems. They go from the goal back to the design. Whereas the the forward direction is science. You say here’s the here’s the setup. Here’s the prediction. And is the prediction realized or not? So okay, yes, it is. That means the model must be good. Uh so what’s the inverse of game theory? Okay. Well, it’s outside of economics not talked about perhaps that much. Uh game theory sounds like it’s sort of everything. Well, the inverse of game theory is what’s called mechanism design. And mechanism design says, “Oh, I want a certain outcome in in the world that this person gets paid that that the wealth is divided equally that you know there’s some fairness or some market that’s created. What game do I design so that that outcome is realized?” So, I’m the designer of the game. I’m just taking the game as given and then looking at what it predicts. Mechanism design has got many pieces too. I work in contract theory. That’s a part of mechanism design. It says, “What if I have two entities interacting? They’re not symmetric. They one knows more than the other and they have to interact with each other.” That’s that’s contract theory. Auction theory is another part of mechanism design where I’ve got a bunch of people coming in and I think of them as symmetric. I don’t know who’s got more money than who wants to bid more than others, but I have this mechanism called an auction that reveals their value. And the outcome is that the person who wanted the painting the most got it. That’s that desired. That’s one desired outcome. Anyway, long story short, game theory is a super rich not so old discipline, you know, 100 years now. Uh that’s that’s continued to evolve and to continue to supply all kinds of algorithmic ideas for those of us who are in the business. So, I’m I’ve been mostly a statistician in my career kind of worried about uncertainty and probabilities and decision-m and uncertainty. But when I go to equilibria and games and or economic ideas, the night gay theory is kind of is part and parcel of the thinking. you’ve said that we need to be thinking about I mean we’ve spoken about incentives we’ve spoken about uh collectives the other big one is uncertainty quantification now there’s this wonderful field uh in machine learning called conformal prediction um it was invented by my professor at university vol and you know so we learned about the transductive confidence machine which oh nice yeah these measures of strangeness so that would be like the distance from a hyper plane on an SVM and you can basically kind of you know um I I suppose calculate something like a P value, right? And have a confidence region. An E value. Actually, an E value. Go on, tell me more. Oh, well, I I don’t want to get into technical talk about E values, but just uh now we’re kind of, you know, Vladimir is is fantastic and uh I don’t know what he thinks of himself as, but I think of him as a statistician. Uh, you know, with game theory background, too, and um, you know, he’s in the school of like the Phil Davids in of the world and the David Blackwells who spilled out of statistics to do all these other things. Uh and so yeah, classically p values were just kind of a oneshot quantity that statisticians would talk about that was like Fiser that said I’ve got a model of what’s going to happen in the world. Uh it gives a probability distribution on the outcomes. Some outcome arrives, it looks very improbable under that model. The model must be wrong. That that’s kind of the p value. And so the p- value is the tail probability. The problem is if you do that repeatedly and you look at maybe the smallest p value along the way that that’s called p hacking and that gives you wrong answers mathematically and then in practice. Uh so e values are different. It’s an expectation of some um uh non- negative random variable or or a non- negative super martinale in more generality. So you’re watching this evidence kind of acrewing and you make sure the expectation of that evidence is less than or equal to one at each step. And then you can think about a multiplicative kind of evidence gathering that if it’s always an exponential less or equal to one then it’ll kind of stay below one and if it’s non- negative it’ll just kind of decay decay away. So under the null hypothesis I’ve got this stoastic process which is kind of decaying away. Well I can look at that at any time and sort of assert that it’s decaying away and I can look at it repeatedly and keep asserting that and I can have control. There’s something called v’s inequality that that Vladimir and others have exploited that says that can be controlled over the entire p you know path of this thing. So now we can do statistics in a new way. It’s called anytime inference. We can peak we can change we can gather new data. We can do this in an up very liberating and Vladimir’s uh yeah one of the leaders of that and EALA is a is a one of those martinales stopped at a particular time by the optional stopping theorem. You can stop it whenever you want. So that has opened up a lot of connections. In fact, our statistical contract theory. Um uh what is a contract? Remember it was like services and prices. Well, the services are uh like evidence gathering. Um and um and and the price also is is part of the it’s it’s a random variable. And um it turns out that we can have an incentive and compatibility in contract land if and only if EV value in statistics land. Um so there’s a nice tight connection between game theoretic probability and the theory of incentives. So I to me uncertainty quantification is rarely just here’s an error bar. That’s kind of classical statistics. And uh it’s more what the context is here. the context might be a contract or it might be uh some other evidence gathering mechanism and this this way of thinking opens you up to a broader class of of of of the of evidence gathering. Very cool. And I should say in your paper you have this figure of a triangle which we’ll put on the screen now but you’re kind of saying there’s you know there’s there’s economics and there’s computer science and there’s statistics. Well these are these are thinking styles I even even call them by those disciplines. So there there was there was a paper by Janette Wing you know couple few decades ago talking about computational thinking. So it says oh computer science is um developed these thinking styles that are more abstract than just computers. It’s it’s uh like modularity and abstractions and APIs and all that and why don’t we teach everybody in all the sciences and all the disciplines to do computational thinking. I think that’s totally right on that’s that’s great. Um but lots of algorithms don’t come about from those kind of computer science principles. They come about from thinking about inferial uncertainty and how do I gather data to you know make predictions about things that don’t yet exist and think about incentives. How do I make sure that you know incentives are in place and I called those two kinds of thinking. One of them inferial thinking so not just statistics a lot of fields have inference in them. and then uh economic thinking. Uh it’s not just econ economics, it’s social scientists of all kinds and legal scholars and so on. When you put those three together, you get a pretty good platform for training of the next generation and a pretty good platform for problem solving of the kinds that we’ve been talking about this entire time. Just one of the fields just computational logs and optimization that kind of gives us LM LM fine great but doesn’t give us any of the context around the LM. The incentives kind of gives you the whole things we’ve been talking about. And then statistics to me is critical. It thinks, you know, about what kind of errors I’m I got to make, how to make sure the data is, you know, that, you know, controlled so I don’t make the errors. And we put the three together. Yeah. They also bring kind of some partners, you know, the the economists talk to the the behavioral, you know, psychologists, uh the computer scientists, you know, talk to the physics people or whatever. Uh the statisticians talk to to to um the legal people, whatever. there’s a there’s a whole sub communities that come together. So to me, if you put on this triangle there and you think is around it, it starts to become a new way to think about uh academia. It’s this is the liberal arts of the era. This is the core. And now my colleagues in the humanities might disagree. The core is, you know, still the humanities, but I just don’t think it’s touching the the core intellectual issues of the era, which is, you know, about data and about compute and all. But I want to put the ingredients in place that those things are thought about in a in in a in a society responsible way. But could you bring this to life? So um you famously spoke about here’s a language model and I’m I’m going to ask it how how confident are you about the answer and it tends to be quite modal. So it’ll either be like you know one zero or or not and like what what’s the difference? Why does the language model not really have any idea about its confidence? I you should ask the language model builders because all they’re doing is predicting the next word and there’s not any thinking about uncertainty quantification in doing that. Uh and you can graft in ideas um you know but they’re not they’re dubious they’re often putting up dubious prior in or uh and and so you can you go to the statistician and and that’s what you know people have done and they’ve said okay I can just treat as a black box and I can put conformal prediction around it. It’s a nice method. Doesn’t require a lot of assumptions. So, yes, that’s true. But it makes a lot of, you know, it’s not there’s an exchangeability assumption. The data, you know, if you scramble it, you get it’s the same. Um, and and so, uh, while I think all that’s really crucial and important, I I tend to think more about the broader context. So um you know I gave an example in that article that you mentioned um of um you know a duck who goes to a lake and that this a statistician duck. So it’s kind of calculated that over the last year there tends to be you know twice as much grain on that side of the lake than on this side. 2:1 ratio. All right. So now the next day I I need to decide and the duck uh which side of the lake I go to and the the basian duck um who has those probabilities would then do the maximal expected value and they go to the left side of the lake with probability one because they’re all right but the actual ducks don’t do that. They go to probably 2/3 to that side of the lake and one third to the other side. They’re hedging. Um but but it’s not just a hedging thing. Hedging would just do occasionally going to the the other side of the lake. they’re actually getting the right ratio. And and and so the explanation is that you weren’t thinking about the context, right, of this uncertainty. Okay? It’s not just you, the individual duck. Probably you evolved in a world where there are many ducks. And if all the ducks went to the same side of the lake, obviously you’ve missed out on a resource. And so is there an algorithm that allows many ducks to cooperate here? Um well, if they all have that same uncertainty, then they can sample with probability 2/3 and go to this side versus 1/3. And that’s actually a Nash equilibrium of the bigger system. All right? Right. So the the right way to think about uncertainty there is that in the context of the population what should be how should I use my uncertainty. Another kind of uncertainty that’s kind of the economic side. Another uncertainty in economics is the one I’ve alluded to information asymmetry. You know things I don’t know and you have expertise I don’t know about but we’re going to work together and I’ll maybe give you a contract a menu of options. Um but even if I interact with you for a while I still might not know. You’ll there’s things you’re going to know that you’re not going to give away to me and maybe you’ll hedge you know you’ll lie a little bit. So I I don’t know about that. That’ll never that’s not just sampling. That’s a different kind of uncertainty. Okay. And then finally there’s what I like to call providence. You know that’s more like a database kind of uncertainty. Um if you um if I want to do a medical operation and um you’re a doctor and you look at the data for people like me uh here’s the you know if you do the operation this way the probability of survival versus this and I look at that I say great but now you tell me oh that data was gathered 10 years ago. And I’m going to say, okay, my confidence interval should go up. All right. Well, classical statistics, you know, could talk about that. In fact, it’ be I’d be more of a basian to think about that, but it doesn’t. It just sort of the data is the data. Uh and and it should be in a bigger system that is data is flowing around. It should always be tagged with metadata about how old it is, and that should be quantitatively brought into the uncertainty quantification. We’re not doing anything like that right now. And so the poor LLMs, you know, which are basically doing none of the above, have to strike out a little bit in all these directions if they’re going to start to do like what what humans do. We are pretty good at um getting these with a little bit of providence. Oh, it’s old data. I discount that. Uh we get a little bit of context. Oh, there’s a social environment here. I, you know, I should just do the same thing. Should randomize. Um oh, there’s some s sampling uncertainty, you know, and so on. We put all that together almost seamlessly. And then we do this in a social context where if I don’t know how to get from here to the other side of town, I will ask someone who looks Danish. I know something about how to gather more data and so on. So the poor LLM has none of the above. And um so what should it say when you ask how sure are you? And all it’s doing to the best of my knowledge is that it’s just well in the past someone asked a human on the internet how sure are you of that equation you just wrote down and someone said some oh I’m very sure because of this or that. And I think it just mimics that those kind of assertions, but that’s not reasoning under uncertainty. And if we did have epistemic um you know quantification, what would be the the main uplift from that? Is it is it about I I know I don’t know something, so I’m going to kind of lean in and try and do more epistemic foraging in that area. Well, again, I think we’re now in statistics land. You know, the statisticians are all about what species are present on the island. Have I sampled enough to know that there’s not a new species? That’s these are classical areas of statistics. Uh optimal experiment design uh you know for that subopul I don’t have enough data. I’m making a bad inference and data collecting in the context of inference in the collect in the context of um you know making assertions and doing that repeatedly. That’s what statistics has long focused on. So I I think give them credit for handling uh a kind of active form of uncertainty reduction. But again for me red uncertainty reduction in the in the large comes about from much broader sets of uh of um components like a market like if I if I and I use example in the paper where I’m you know I want to have a restaurant like this for pizza and I need tomatoes and so if I had to forage for tomatoes every day it would be pretty uncertain whether I would have pizza that evening but because there exist a market where someone else did the foraging there’s a stable amount of tomatoes every day I can I can build my restaurant assuming that that’s true that my uncertainty for finding tomatoes went down. Therefore, I can build on top of that and do other things. Markets mitigate uncertainty and they they don’t do it because someone designed an optimal experiment design or you know ran or did some multi-arm bandit you know not directly uh but because the market did try various things out there’s incentives for people to explore and exploit. Professor Jordan, it’s been an honor having you on the show. Thank you so much. All right. It’s been my pleasure. I’ve enjoyed talking to you.