Michael Nielsen On Hyper Entities Tools For Thought And Wise Optimism

TITLE: Michael Nielsen on Hyper-entities, Tools for Thought, and Wise Optimism CHANNEL: Foresight Institute DATE: 2025-08-07 ---TRANSCRIPT--- Silicon Valley relies completely on sort of there being a supply of hyper entities around it. Once somebody’s had the idea, you know, it’s very easy then typically for it to be copied and for other people to sort of act on it. I’m very interested in this what I call vision papers. So when people do sketch out, you know, molecular assemblers or they do sketch out quantum computers or or whatnot, those papers do often read quite strangely and they sort of stick out a little bit in the scientific literature. They’re not normal science. They’re proposing some imaginary future object and saying it will have these properties. The difference between now and 2020 in terms of possibility of AGI is not that large, but the the difference in belief is absolutely enormous. Okay. So today’s guest is Michael Nielson. Michael is a scientist and a writer. I think maybe best known for his work on open science, quantum computing, and tools for thought. He’s been a leading voice in I think reimagining how science is done in this age. And Michael, you you have this like really broad background and scope of things that you worked on obviously. And I do love reading your blog. Um it’s a very broad range of writing. And I think in this conversation what would be really interesting is like exploring and connecting much of all these various topics that you’ve written on to this idea of existential hope and just like positive futures in general. So maybe we just dive in and I think start with one of the central concepts that when uh I found it in your work I thought was just this really interesting way of putting it uh which is this concept of a hyper entity. Could you maybe explain how you came came up with that concept and um you know dive into what it is a little bit like it’s not just like a meme or something for example. >> Sure. Actually it’s funny it’s it’s a little bit embarrassing. So I wrote this long piece 40,000 words about how to be optimistic in the face of artificial super intelligence and that’s a very heartfelt very serious piece from my point of view like really mattered to me but there is this little rant in the middle of it about these hyper entities and it really was it was just expressing sort of irritation and I’ve been a little bit prized shocked even a lot of people have told me that that actually that’s the piece of the essay that they responded to. It’s this kind of an aside from my point of view, but it’s really I mean it’s riffing on some some older notions which I didn’t feel quite were the right term for what I I just wanted a term for an imagined future object, some sort of hypothetical future object. So artificial super intelligence or quantum computing or in the past you might have had things like actually universal programmable computers or heavier than air flying machines were sort of examples of from the past where somebody has imagined a future entity um uh and and that has then sort of served as essentially sort of a design goal um and I’m particularly interested in the question of or sort of the aspect that I’m interested in is in some sense the extent to which imagination was required to do the the sort of the design work. It’s not I don’t know a notion like a universal programmable computer like seems very obvious today because our culture carries it but at the time it was a shock and required enormous imagination and enormous understanding to come up with and that’s sort of the those are the the connotations that I was particularly interested in in carrying. Oh, actually a good one. Uh, actually for the the foresight institute, which is associated with with many, but I think in particular, one of the founders of the foresight institute, Eric Drexler, has this notion of molecular assembler. Um, and that’s a great example um, you know, of something that required a lot of depth of understanding and imagination to come up with, but after the fact seems pretty pretty obvious. said, “Well, it’s not just Eric who did that, but I mean people like um I guess Heinland and and Fineman are sometimes given sort of partial credit, but together you know sort of this group of people had this this very deep insight and then it serves as kind of an orienting vision. Sorry, it’s a very long answer, but I it’s not a context. I think yeah I think the only thing that I would have heard someone like say similarly is like a future artifact or something almost like that’s something that when we’ve done workshops on like world building and these sort of things it’s something that that seems like the term that most >> sure I guess there’s a couple of other terms I mean Bruce Sterling has this nice term spyime which he uses to mean something quite similar it’s got a little bit more sort of it’s an object which has a lot of knowledge about itself Nick has this term hypstian which is it’s kind of an alternative The term superstition, it it’s about it’s really it’s sort of a short phrase for self-fulfilling prophecy, something which becomes so attractive to in existing institutions that it then sort of forces itself into existence. But those terms, they’re slightly different in in the sense of the connotations or the particular focus there. Although they’re talking about future entities, they’re not really talking about the particular actions that you do with them. I’m interested in sort of the way in which these these kind of entities, they tend to carry new verbs with them. So the idea of a programmable um computer, the idea um that uh you specify a set of instructions uh and a uh molecular assembler will be able to build any uh uh type of material object within you know there’s going to be some parameters but very very broad. So you sort of have these these new kinds of verbs or affordances in the the design terminology. So it’s very much sort of an a design point of view. >> What was the first term you you said there? Spy. >> Spy. >> That’s from the the science fiction author uh Bruce Sterling. He actually, in fact, it wasn’t in a science fiction book he wrote it. He wrote it in what was it? I can’t remember. It’s like a long essay about um really sort of the design of sort of in in the internet of things was kind of the the context. Uh it’s quite a nice term, but it doesn’t seem to have really caught on. I’ve heard a few people use it, but uh >> yeah, >> not like hyper entity apparently. That’s the one that um well I think that if I don’t remember correctly also or if I remember correctly from your post it was also you were sort of suggesting that maybe Silicon Valley tends to undervalue hyper entities. Do you want >> Sure. I mean undervalue not exactly they rely completely on sort of there being a supply of hyper entities just the way in which the um uh you know the economy works of Silicon Valley. there’s no way of building a moat around it once somebody’s had the idea. Um, uh, you know, it’s very easy then typically for it to be copied and for other people to sort of act on it. So typically it’s coming from other parts of of the world, academia and art and sort of the penumbra around them. I sometimes talk about paracademia. Those tend to be places where these ideas come from. Actually, the the examples I gave before, you know, molecular assemblers and artificial super intelligence and and universal computers, they all come out of academia or the little area around. There’s examples like say virtual reality, which actually comes much more out of the artistic community. What’s going on there is that sort of the the economy so to speak in in those cases, you know, it’s a reputation economy which is built around citation or attribution of ideas. And so people somebody like uh David Deutsch um you can really build a a reputation um and and a career around an idea like quantum computing. But the economy in Silicon Valley works differently. It’s about around equity and around sort of ownership of the means of production. And so you you sort of have these two different systems that are coupled in some way. Sort of the the ideas coming out of academia are often used as almost sort of a a feed stock for Silicon Valley. But it is I think somewhat somewhat separate. The the way I tend to think of it is that often in Silicon Valley ideas are they’re very instrumentally useful and and understanding, but they’re done in service of whatever the product and whatever the market is. Whereas in academia and also to some in a slightly different way in the artistic world, the ideas and the understanding are fundamental. That’s the object. That’s what you’re you’re actually trying to do. And your ability to sort of execute to build the apparatus or whatever it is, that’s a secondary thing. that’s instrumental to sort of the the the the fundamental goal which is to obtain that that understanding to obtain the deep new ideas. Yeah. I mean you have these two as I I I think of them as two coupled economies fairly weakly coupled like they both have their own reasons for existing and I don’t want to get into any sometimes people will say oh you’re sort of making a judgment my personality is such that I’m happier in the sort of the artistic scientific realm but of course the world is made up of things which which come out of the other the other realm. So they’re both very very important. >> Yeah, we had we had Ed Finn on who was the editor of the hieroglyph book that came out where where they like connected scientists with sci-fi authors to like try to come up with scientific artifacts I guess like future artifacts. Do you think like do you see um do you see sci-fi as a potential tool for like hyper entities or do you think that it’s like maybe um too cliche? I mean historically yeah people love to point to the example of Arthur Cy Clark and satellites geocynchronous satellites which actually wasn’t his idea although he popularized it and there are a few examples I I constantly find Verji the science fiction author really remarkably preient and how he viewed the future I think I actually see almost more connection to to design than to science fiction sort of this idea of developing new classes of object with new types of action new types of interface um is is I sort of think of it as more a meld of uh scientific understanding and that the fact is I mean nature is just so much more imaginative than we are. Um and so a lot of the most remarkable things that have been discovered you think I know general relativity or superconductivity or ideas like this the fact that light is an excitation of the electromagnetic field like these are mind-boggling facts just utterly shocking so much more interesting and surprising than anything a human being has has ever sort of imagined denovo so in that sense I think you actually sort of need that marriage of very deep insight into how the world works with again sort of a design point of view and science fiction writers to some extent have that. So of course do designers but the science fiction writers themselves I think often don’t have quite enough sort of depth of insight into the world. They’re stuck recycling ideas that they’ve seen often very effectively and they can sort of see all sorts of interesting consequences. It’s not quite the same as as invention. >> Yeah. Um, do you think that there should be more like conscious effort for example from Silicon Valley in trying to like design hybrid entities or like come up with new ones to generate excitement or like a direction >> really I mean they’re doing what they do. I know I think about actually a good example I forget which lab it was Palmer Lucky the founder of Oculus um you know was in I think it was at USC uh a basic sort of science lab that was fairly basic science lab that was just doing experimentation with sort of very simple prototype uh VR systems and so that’s in my opinion an an example of sort of the supply chain working reasonably well you’ve got this fundamental work being done which in its own way doesn’t need any justification but in practice it does feed And there’s always this sort of interesting friction between the the two worlds because they have two such different fundamental objectives. But I I don’t think particularly Silicon Valley needs to needs to change what it’s doing there. There is this I suppose this sort of interesting question about public goods production which is that the nature of ideas and understanding is that they most naturally want to be public goods. That’s in many ways best for our society. you do I don’t know it frustrates me a little bit to look to imagine you know what’s in the internal mailing lists of companies like Google and Facebook and Microsoft and actually for that matter now companies like OpenAI and Anthropic that there must be just an astounding amount of understanding that is hidden there that would be tremendously valuable if it was made public for the world but is going to be forever forever lost. They’ll bet that most of what we know about distributed systems is basically hidden inside old Google mailing lists, but it’s not, you know, it’s not in the company’s interest necessarily to be to be releasing that. So that’s one sort of point of frustration, but but I think it’s intrinsic to the situation actually that the companies do have a lot of vested interest in keeping that stuff private. Unfortunately, I don’t don’t know how to solve that design problem. Yeah, I think on the Palmer lucky point, he I heard him say that he got his all his best ideas for like weapons because he works on on that now from sci-fi, which is interesting and like maybe not the the way uh that I was hoping people like get excited from reading sci-fi. But yeah, that’s why I feel like with the hyper entities terms term there is something in terms of like if we actually want to think about how we can steer towards like not just a future but like a better future that there seems to be something in terms of like we should maybe put more effort into how we design the hyper entities that actually end up having like obviously it’s like a very complex and random thing maybe what becomes the hyper entity. >> Yeah, it is a bit strange. um you know and this is actually I suppose connected to to Nicolan’s point about hyperstition the extent to which different things become popular or become sort of seen as targets in some sense the difference between now and 2020 in terms of possibility of AGI is not that large but the the difference in belief is absolutely enormous and I think about say you know pick a sort of a parallel world I don’t know crayonics say you imagine that sort of the same degree of belief and capital had sort of been infused there. What would that field look like now? It would look probably very very different. Um so you still have people sort of working on it but just not at the same scale. You don’t have thousands of people um sort of coming into I don’t know Hayes Valley or wherever um uh trying to work on this. So that degree of sort of belief and self-fulfilling prophecy is actually pretty important. And it seems it’s weird. It’s a little bit random certainly. It’s very I mean it’s so influenced um by single actors. you know if uh tomorrow you know uh somebody comes out and announces that uh you know they you know they don’t think that that open AAI should have a down round some very you know important investor who participated in prior rounds that would have a very large impact and it it would it’s potentially just down to sort of decisions by by a few people um you know don’t want this to happen if Taiwan was was invaded and again that that decision can just be made by you know a small handful of that would be also have sort of an enormous impact on this degree of belief. So, so it’s sort of funny how though that works. >> Yeah, it’s hard uh hard to steer I guess or like hard to design ahead. >> Well, it certainly it does seem to it’s certainly possible to steer, but it’s just so dependent on sort of contingent actions by um uh it’s some interesting combination of sort of power and sort of truth is important there. Power does matter a lot in the short term. Truth matters I think more over the long term. If um AGI is not going to happen through anything remotely related to LLMs, then you can pour as many, you know, trillions of dollars into it and it doesn’t matter. That’s just a fact about the world. I’m not saying it’s true. I’m but I’m saying that that’s a, you know, a potential way of the world uh the world is sort of make 2 plus 2 equal to 5 for a while with enough money so to speak, but over the long term reality wins. >> That’s that’s true. reality which um like my my last question on the hyper entities point I think would be um cuz obviously I think >> AGI is like the hyper entity of the moment at least in you know our our sphere maybe but are there any other are there any other ones that you’re personally excited about or that if you think about like what would bring potentially the most value to the world what uh what would be such >> I mean it’s interesting you you you say that that it’s of the moment I mean in some sense renewable energy is actually still more important I think it’s what is its uh profile fuels are roughly a $3 trillion industry, much much much larger than than what is conventionally called tech. And so the ability to replace that uh by something else um is in some sense sort of more important. And the idea uh you know sort of cheap photovalttaic power and cheap batteries remain these kind of orienting visions for I mean they have been for decades although I think probably a lot of people didn’t really I mean some people still don’t believe in those actually. Just to go back, can can I can I get you to repeat the exact phrasing of the question? Uh >> oh, I I think the question just like are there any other hyper entities that we should you know that you think would be most valuable actually right now? >> Certainly I I am very interested in and excited by um uh sort of design mechanisms for solving public goods problems and collective action problems. So ideas like assurance contracts, Alex Tabarox’s idea of dominant assurance contracts, the Victory Clark Groves mechanism for which um Glenn Vile and Zoe Hitik and and um Vitalik Bhutan have been working on with quadratic funding. These sorts of ideas, they’re very interesting. If you can solve the public goods problem or collective action problems, that’s like just totally transformative for civilization. It’s probably related to it’s not the same as but you know there’s I mean these c centuries old dreams of global governance working and we’ve never really figured out how to to do that. We’ve sort of made some I suppose things like the nuclear non-prololiferation treaty and the Vienna and Montreal protocols are very important progress that seem somehow related but gosh our governance mechanisms seem primitive >> actually that’s I think it’s a really interesting point because um when with the existential program I’ve done a lot of workshops and stuff on what what future people want to see and then also like what what do you think is the biggest blocker and I think a really common theme is just like human nature or you know these sort of things and so yeah >> so last year I read uh Tom Holland’s book Dominion which is I mean he can describe it in many ways but to some extent it’s a history of kindness it’s a history of charity and he’s particularly interested in the question of what influence did the Christian tradition have on our modern notions of what it means to be a good person and he claims I think sort of fairly plausibly that in fact they really contributed a lot to people valuing kindness and valuing charity. Obviously people have always people selfishly value kindness. We like it when other people are kind to us, but they don’t necessarily hold it up as one of the chief virtues. And he sort of will claim that ideas like, you know, turn the other cheek and and love thy neighbor as thyself, which come out of the Christian gospels, that one of the most important impacts of Christianity has been to really amplify those ideas not just in the Christian world, but in fact in the secular world. He traces how it really incredibly influenced the enlightenment and also other cultures. So you can go to, you know, many very non-Christian countries and they actually hold sort of views about kindness and charity that that Holland claims uh you can be traced back to to the gospels. So I think that that kind of thing I mean that’s I don’t know describing it as a social technology is kind of a bit silly. What’s the right way of thinking about it? Something like I mean it is very interesting the extent to which those notions are constructed out of stories. They’re constructed out of myth and transformative for for civilization. I don’t know that you can get things like human rights. I’m reading about history of human rights at the moment. I don’t know that you can get things like like human rights and and sort of the modern you the suffragette movement and and the civil rights act and things like that without those those >> Yeah. I feel like I I feel you can almost think about them as like social technologies. It’s I think it’s especially interesting now or it becomes very evident in terms of like this is what we want to instill in our AIS now or like we want to make sure that this is >> we have take off with these values not other values or like you know it’s >> um well so I think that a lot of your writing is about this you’re obviously very excited about technology which I think you know at forite we’re obviously very excited about technology and like that we could you know if you read Drexler there’s engines of creation uh back in the 80s. It’s like all these exciting things that we could do. Um but then there’s also like the point of all these very scary things that also could um be achieved with these technologies. And you wrote the the post on um how to be a wise uh optimist is the the term um technology. Um could you maybe help us unpack a bit what uh what that means to you and um also how yeah why why it’s important maybe? >> Sure. Uh I suppose I just sort of amused almost by the growing use of the term optimism in uh discussions about uh technology and the future of technology um and what seemed like a a very bad misuse. you if you’re a patient in, you know, goes to see your doctor and you’re diagnosed with cancer, the optimistic path isn’t to, you know, ignore it and to, you know, pretend it’s not happening and to continue on about your your your life and just hoping that somehow spontaneously it will go into remission. The optimistic path is to really take it on board and not to get depressed. that would be the the path of pessimism, but it it would be to take it on board and to ask what can I do about this situation? And so a lot of sort of what passes for optimism in tech seems to me like a foolish optimism where it’s like no no we’re not going to listen. We’re not going to think at all about the problems that that these AGIS could cause. In fact, it’s even it’s actually quite pessimistic. It relies on you believing that they’re not going to be particularly capable. If you look at the people who believe that they’re going to be the most capable, they’re the most optimistic about capabilities, they are the people who are actually the most worried. So sort of the actual optimists in some sense about capabilities are very worried. They’re the people who are facing up to the fact that there is this sort of very difficult diagnosis. Um and and I think the the sort of the optimistic response, the wisely optimistic response is to ask what can we do about it? That doesn’t mean you necessarily going to find an answer. You may just it may be terminal but at least you can engage seriously with the actual state of affairs. Do your dandas to uh uh to to find positive things to do. doesn’t mean that you need a solution, but even just the ability to start working on on tiny little things is >> that’s actually yeah that’s really true that the ones that are the most optimistic about these technologies are also the most scared or like it’s like the the transhumanist point or I feel like that’s definitely an observation that I’ve made that people that are like the the people that almost started thinking about all these existential brisk stuff like Nick Borm or like an or these people they’re they were transhumanist, they got like really really excited about the future and then they were like, “Wait, what if all these things get in the way of of making that future happen?” And so I feel like that’s how >> yeah, we ended up here. Yeah, I think that it’s really interesting also because there’s the point of like shoot you’re almost like shooting yourself in the foot if you don’t if you don’t take into consideration the potential downsides of these technologies as well because you probably might get a lot of opposing u views against you basically. >> Well, I mean I mean I mean always in sort of history the first jet a loiners um a lot of them crashed that ne problem needed to be needed to be solved. detector de Havland went out of business uh because of it with their comet and uh I mean there’s so many examples like that the early refrigerators leaked ammonia gas uh and killed people. They actually needed to replace it. They replaced it funnily enough by chlorofluorocarbons um which was a big step up. It didn’t immediately kill people but of course did have this other problem um uh and sort of that that pattern of actually be honest about the problems and figure out how to fix them is foundational for a good well foundational for existential hope. >> Yeah. I think in in I think it was in this same post you also write about you know the fact that advances in science and these things well that’s what brings us closer to the truth or like we know more about the world but truth is like dual use is that is that still your your take like and do we then does that mean we want to get as close to the truth as possible or what because I think for example if you think about the existential hope point when when I ask people what is like the most existential hope future you can imagine it’s often like knowing or learning as much about the universe as possible is one of the I think most common things that people will say. But yeah, what’s your take on on that as like our ultimate goal? >> I don’t know. Uh I suppose um you know that seems a little too polyiana to me. Um certainly the way I I look at you know a good example I mean I spent much of my career as a as a quantum physicist and um um you know there’s this sort of astounding set of of ideas from about 1900 to 1925 1926 uh were developed which is you know it’s the most fundamental things about the way the universe works and it’s very important then for later things you know if we want to understand proteins or DNA or modern semiconductors and so any other modern material science, but it’s also part of what led to the understanding of nuclear physics that resulted in nuclear weaponry. And I don’t think you you don’t really get that alakart, unfortunately. It’s sort of it’s all or nothing. There’s no once you’ve got quantum mechanics, you’re going to get all of those things. And this seems to be it’s just an empirical observation. I can’t really prove it, but so often sort of any deep understanding of of reality ends up having both many positive uses and many negative uses. I think it’s something there’s something funny going on where we I don’t I don’t really understand quite why this is this sort of intrinsically dual use nature of deep understanding. that I mean a really nice example there’s this famous declaration by the mathematician Hardy that he couldn’t think of anything more useless and more beautiful than number theory which is much of what he’d spent his time sort of working on. Um and then I mean not that long after his death GCHQ and the NSA discovered that uh number theory could be used to develop cryptographic systems that you know have all these sort of interesting military and other applications. He was kind of wrong. you you took even the most apparently useless sort of set of ideas that seems very beautiful but very disconnected from human reality and it turns out it’s actually very connected to to human reality. Gosh, I mean much of the reason for the NSA and whatnot funding of quantum computing is because they want to be able to break these codes based on funny facts about factoring of of large numbers into their their prime factors. um Hardy I don’t know what he would have made of it but again it it’s just like you know these very deep facts somehow then show up in very mundane concerns in in very important ways and sometimes in very dangerous >> it’s uh definitely dangerous >> it’s pretty strange actually I mean you see this so much there sort of there’s sort of thousands of examples like this every once in a while I’ll see going the the remanita function in an unusual place or things like this it’s just so these things that seem so obscure and and so disconnected and they turn out eventually to be um to be very important in the everyday world. Well, I guess there was something very like it’s easy to get excited about that or like you know it is nice when you’re able to make connections and connect the dots and as a human and like you know expanding maybe you know the the the Star Trek vision for example seems like the most exciting vision that I feel like anyone has been able to come up with so far at least that the most well I’ve talked to can get behind. >> Uhhuh. >> And that’s so much about just like continuing to discover the universe and so on. >> Yeah. I don’t know. I mean, I didn’t I have not watched Star Trek. Um uh I’ve watched probably a tiny handful of episodes. I think only from the original series have I watched a single complete episode. Um um something that’s funny about a lot of these visions, they’re often a bit strange from sort of an emotional point of view um or an artistic point of view, they don’t they often don’t I made this point before about um the spread of kindness as something to be valued and as something that then eventually gets built into sort of governance. And I think you often don’t see so much of that kind of thinking going into visions of the future. like what other sort of fundamental emotional stances will people sort of create and start to spread? Um uh and I don’t have an answer to that question. I’m just pointing out that a notion like, you know, love your neighbor as yourself is a shockingly imaginative idea to have and then to spread and evangelize and and sort of just wondering sort of what other ideas like that which have to do with the sort of I mean that that’s a a case of basic stance towards the other. Um actually I suppose I mean some sort of modern ideas about veganism and you know who should be treated as a moral patient and these kinds of questions those are there’s obviously very interesting and what not but I I think notions around identity are likely to fracture an awful lot in coming years they’re already sort of fracturing in interesting ways in human in human beings but they’re going to fracture a lot more. I mean ability to merge and refactor and change and distill out and to do all these kinds of things with artificial minds. The lack of there’s something sort of very essential about sort of the boundary around a human being but that those boundaries are just going to be utterly changed. It’s not even it’s going to be very hard to say what it means for there to be a boundary around them. I don’t I never know quite know in the background when using Claude or whatever am I using the same model as I was using last week to what extent are they actually make you know sort of slightly tweaking it all the time. Um and of course there is that that funny feeling I think many of us get where you keep starting new instances and it’s like you’re sort of starting a relationship again and again and again and again with this thing that has some human characteristics. Um but you you know it’s a it’s a very strange experience. It’s a very human experience in some ways, but it’s also a very inhuman experience. Sort of this, you know, reset on identity over and over and over again. It’s the groundhog day of personalities. >> Yeah. I I think on the on the values point, I think that’s definitely one of the it’s so underexplored basically in in especially in the current context I think that we’re in. Um, and I think that that’s I, you know, with Foresight, you very much meet with people that are very technoptimist and like a technologist seen as a, you know, potential tool for a lot of these things. And I do think that there are like potential exciting applications of just like moral circle expansion also a lot of challenges obviously but through new ways of being able to communicate like neurochnologies and these sort of things which is you know interesting and maybe to be able to communicate with someone on just like a whole different level of sort of granularity or like nuance um could be like a new moral revolution or something like that. that would be best positive best possible outcome. >> Certainly seems like it’ll be well I’m not sure it’ll necessarily be best but it’ll certainly be an expansion and sort of a change in really interesting and challenging ways. Yeah, I think that unless you have anything else to say on that, I’ll I’ll jump to another point which is a bit like the the the existential hope. >> Actually, can I just mention mention one thing which is you know sort of when this kind of point is made people will very often name David Pierce’s hedonistic imperative and these sorts of ideas and I find it interesting that they always name the same example. That’s often, you know, when you sort of notice that happening in any domain, it’s often like, well, actually, not much is really going on over here except that like, and that’s very interesting, but it doesn’t indicate to me it’s a little bit sort of underexplored. >> Yeah, it it’s interesting if it’s underexplored or is there is there nothing else there? But I do feel like there is something there. >> Uh there’s no way there’s there’s uh nothing else there. you think about I mean even just the way in which I don’t know capitalism has for example modulated our our uh uh our values um uh in in so many different so many different ways and so sort of you know any changes to that are going to to to to modulate our our values I think in very interesting ways there’s just there’s too much moral change over the last you know gosh there’s a I read a uh what is it a little New York Times piece from about 1905 five which recorded that I’ve forgotten the name Otto Benjo I think it was pygmy had been brought from Africa and was now being housed in the Bronx Zoo and like you think and and the article it’s not exactly approving but it’s not exactly shockingly disapproving either and can you imagine that that headline now it’s just it’s impossible to imagine >> not even on the you >> no yeah yeah it seems it just seems ludicrous and yet this is it’s 100 years ago So sort of th those differences. It’s funny how slow moral progress and how stop start it seems and yet as soon as you go sort of over a hundred years I know I’m using the term progress kind of a a wig view of history but there’s definitely change really big changes and and people like Martin Luther King or Gandhi or Mary Wencraft and people like that are you know sort of make such enormous differences in the way we think. Yeah, I I was thinking about this just because I was reading a book about women working in a hotel 100 years ago and I was like we get to work you know because and that was like the exciting thing then and so yeah we definitely you know when people mention only one example also >> it is interesting I think with like futurism in general I feel like this has been the case for like quite a while or at least since I became interested in it and and when I’ve talked to people like for example Christine our co-founder She’s been around since you know with Dre before since the ‘8s and it’s the same things being discussed sort of obviously things will feel more like for example AI is it feels like it’s actually coming now. >> Yeah. >> And she said that it feels different also compared to cuz they were obviously talking about nanotech in the 80s and she said that well this this is we can see the progress just much more clearly now. But it it is interesting I think with this that’s why I think that people connect to the hyper entities idea so much because there is just like the the vision for the future is a little bit outdated. I feel like some people can get behind it and the you know the the Silicon Valley um mafia doesn’t really mind that type of like 80s vision of the future but it doesn’t feel like it speaks very broadly to people like more global scale. Yeah. I think that’s just like even if you if you prompt your LLM to like create a v vision of like 2035 it’s going to be you know silver clothing and like white houses and it’s this like oh because that’s how people imagined the future in the ’60s or something like that. >> It’s something I suppose I mean I think I mean this is something where a lot of the best science fiction writers are doing really interesting um things. I love um Carolyn Cher for example, you know, she has this just beautiful ways often of thinking about identity. Mashalitzi has a really nice review, just a very short review of of Cher and it just points out that, you know, most people think of an identity crisis as being, you know, am I in the right job? Cher’s characters, their identity crises are about am I in the right species? Am I sentient? These kinds of questions. And it’s a good description. She’s very good at thinking in in that kind of way. and sort of just drawing it out and expanding your own consciousness of what’s possible. It’s also it’s something I love about uh some of Ted Chang’s writing which tends to be sort of much more near future, but again he’s a person that has this very sort of expansive in some ways. Um uh notion of what being a human being is about and or what being a sentience is about. Uh so it sort of tends to be a bit more interesting than just being about technology. It’s a funny funny way of putting it. I suppose I think of sort of technology as the interface between humanity and the universe. Um, so I’m putting a lot of focus on humanity there and not so much on the universe. It’s kind of all three of those things are are very very interesting. I suppose maybe as a writer you you maybe need to focus a little bit on on >> Yeah. I I I wasn’t I’m not familiar with this writer. I feel like I have to >> cher >> Kan Oh, yeah. She won she she’s sort of won all the awards and her book uh Saitine uh is I think her probably her best known book. I’ve probably only read maybe five of her books or something but Saitine is is one of my favorite uh novels. Is very slow to get started. I almost gave up several times in the first 150 pages and I’ve probably reread it half a dozen times now. It is that’s a very interesting exploration of what it means to clone an identity and sort of it explores that through multiple instances in very different ways quite a remarkably insightful way actually her background is is really interesting she was I believe trained as an archaeologist uh and it really gives her uh you know she’s not a technologist she has this instead she’s steeped in this other very deep tradition um of um how people enter the world and uh there’s also something very interesting just sort of in the aesthetic. Well, it’s it’s more than the aesthetic, but there’s something very interesting going on that I don’t know enough about archaeology to understand, but you can feel it. It’s like this person has a depth to to them that is very very interesting. >> Yeah, that that’s interesting. It must be like a very she she knows what it’s like to explore past civilizations at a very physical level. >> Yeah, I guess I guess so. I don’t know how much field work or whatever she did. In fact, I’m not even 100% sure where I got that information that she was an archaeologist. Uh but uh or she was trained as an archaeologist. Uh but it does it does it’s very consonant with uh with what it what it feels like to to to read her. >> We’ll we’ll link it on the to this episode so people can explore it, myself included. Yeah, I think that uh actually one of the I’m gonna move us on to the next also, but just while we’re on this actually I think my favorite sci-fi TV right now is the Murderbot series which is available on Apple TV. Now it sounds like I’m advertising them, but it’s I um I was just really excited to see something so like everyday light and like um relatable in in terms of sci-fi. Yeah, it was it’s just I recommend it basically for anyone. So, yeah, I think what I wanted to also make sure we have time to talk about is just I guess it’s kind of the point that we it relates to what we’ve already touched on, I think, but I know that you wrote about like that for example breakthroughs like language don’t necessarily come from like goal-driven processes. So, it’s a little bit of what we about before. And so if we’re trying to think about like we want to invest very wisely for the most optimistic or like the most positive future, do you think that we should like try and engineer for like a specific outcome or are there any like underlying systems instead that we should focus on like tools or institutions that will like open up the possibility for positive surprises. So like is it this very concrete ideas that we should aim for or is it more like creating like a fertile grind ground of discovery that we should invest in more? >> I mean I think you need to do both those things. My personality is to prefer the the the more exploratory approach. I know one of my favorite facts is Charles Darwin didn’t go on the beagle to do biology. He went on it to do he thought he was going to be doing geology with a little bit of biology on the side. talk about sort of undirected exploration paying off and people like you know uh Newton didn’t have a goal of inventing modern science like he couldn’t have he discovered the goal um Einstein couldn’t have had a you know he didn’t write a grant application to to to change our notions of space time energy mass he uh uh uh was was just exploring um and sort of those are things that he discovered after the fact so that’s my personal prejudice is sort of in those directions actually I love the the language example that you bring up, nobody again that wasn’t the result of a grant application and you know a big uh uh uh uh you know push by the NIH or whoever NSF to do that. It’s it’s something that that that emerged. But by the same token, um I think you said in some of the background materials you sent me before, you were pointing at things like the uh Apollo program as like really interesting kind of unifying visions. And of course, you know, I suppose in a really interesting way like your question is about the value of coordination versus exploration. So, you know, if you’re going to coordinate, if you want to build, goals are very useful for for coordinated attacks on problems. You know, you build LIGO this way, you do the Apollo program this way, you build the LHC this way. Uh, an exploration is more sort of something which is done by individuals or small groups. Part of the reason I maybe feel passionate about defending the exploration is my sense is that the current time in history is a little bit anomalous in that sort of the tendency of bureaucrats, the tendency of leaders is to prefer a goal oriented approach. A very natural thing to want to do. You can make the big announcement we’re going to cure cancer and da da da da da da. And so you can sort of direct a lot of money in that way. Whereas sort of this undirected approach where you don’t actually know what it is that that is going to be found is harder to defend. It’s just so much more hunch driven and in particular sort of in very often in very illeible ways. So I feel like it sort of it needs a a bit of a constituency a bit more. It it is a bit funny that in fact we’ve we’re in a situation where that’s our society has sort of adopted that Bush wrote this famous memo um in the United States which led to the foundation of uh the National Science Foundation in the United States and a lot of sort of copies in many other countries but one of the things that that that memo I believe it’s the first place that the term basic science has ever used and that was kind of a coinage really as kind of a part of a sort of a he was looking for a weapon to win a political argument in favor of this kind of basic undirected exploratory research. And he won that argument really comprehensively, which is I think why sort of there is as much support for uh for basic exploration as there is now. But the natural pendulum is is in the other in the other direction. That’s kind of the the high modernist approach is to you know map out where you’re going to go and whatnot. my my uh friends Adam Marblestone and Anastasia Gamik uh you know are running Convergent Research which builds these focused research organizations as you know um um and they’re lovely and I I am very glad they’re they’re doing that but I I don’t want a future in which everything’s a fro either. I think that would be bad. I think they’re also very aware of that, but it’s it’s hard for, you know, what’s sort of self-limiting or self-regulating, like what sets the scale. I don’t actually don’t actually know. Certainly, that kind of model is very attractive to to political. >> Yeah, I think as a funer, it’s definitely very scary to to fund the exploration because you may end up with nothing. >> Well, in fact, you will end up with nothing almost all the time. Although I I mean I think if you look at you know as I say sort of historic examples uh you know how much uh sort of future work like that is justified by Max plank inventing quantum mechanics or Albert Einstein inventing general relativity or Darwin inventing and sort of so on and so forth. muricy and and radioactivity all these kinds of things um uh sort of deserve some well yeah it’s it’s really this point I suppose David Deutsch makes it I think quite nicely you can only plan you can only set goals if you know what’s out there and yet by definition we actually haven’t seen most of the universe we we most of the sort of parameter regimes are just are still completely unexplored and so you can’t really plan well in in those places and yet historically they’ve been the most exciting places to go. >> Yeah, I think um I want to explore something that touches on this which is you did write in uh so I think to sum up that point it it feels like it’s pretty clear that both are needed like we need the the exploration that probably takes some people to fund that that have the guts to do that and we need the the coordination but I do know that in one of your posts you actually said that maybe we should consider having like this serious discipline of imagining the future and that could actually be really important and so that would be a bit fun to riff on. I don’t expect you to have like perfect answer but um if we actually think about doing that in practice and I think it’s >> I think can I just clarify something um I think I mean I made something very specific about that I mean people have been imagining the future sort of seriously for well well over a hundred years I think about I don’t know Herman Khan and people like this and well many many others I was referring very specifically to this task of conceiving of well new hyper entities in particular sort of taking this design point of view which says the the most interesting thing you can do is essentially find new verbs. And so that was I think the thing I was probably pointing at the notion that I don’t know the Victory Clark Groves mechanism. So this idea that if you change the way in which you vote or allocate resources into this kind of quadratic model, this will help you solve public goods problems. Like that’s a that’s a sort of a shocking set of new ideas. That’s an example of sort of some sort of imaginative contribution. That’s that was the point. It wasn’t about predicting the future. It was about imagining the future. I think that’s kind of the the the the core of it. And so much of the work that’s been done on sort of what’s been called foresight studies or futurism and various other labels, it’s often about prediction and sort of some degree of imagination, but it tends to take kind of the extent objects as being they’re sort of an input to the process. They’re not really sort of an object of the process. And I’m talking about making them more of an object. Hopefully that’s clear. >> Yeah. No, I I think it’s definitely clear that it’s the imagination, which I think is really interesting because I think that that’s often what gets taken less seriously or it’s it’s easier to to wave off uh and maybe harder to see like short-term value in some of those. >> Well, I think I mean the question of sort of how you validate the imagination is is a really interesting one. So um I one of my favorite examples is Alexa Katayv has this incredible idea of a topological quantum computer which is sort of it’s founded in a state of matter that naturally wants to store quantum states and where if you sort of twist it around in the right way that will result in in quantum gates being applied to it. this I mean this sort of should sound ludicrous and and almost incomprehensible but it’s such a feat of imagination to conceive that this would be possible and then actually technically to come up with models in which it’s very plausible but it’s those models and then ultimately actually building the systems which is you know that’s what validates the imagination there I think something like um actually um uh the assemblers um you know the book um well first engines of creation but then the the PhD thesis nano systems um you know did this all this sort of interesting validation work showing that it was at least somewhat plausible um that you could you could build these systems and what they would be would be used to do. So there is I don’t know some notion somehow of validated imagination there if that makes makes sense. Yeah, that’s true that it’s like scientifically grounded or like maybe it’s possible, physically possible as probably >> it’s physically possible and also and you sort of you want some sense of and there’s something essentially new here like I gave this example of the topological quantum computer and there from the point of view if you told a quantum physicist 50 years ago that it was possible to store quantum states and protect them in these macroscopic phases of matter, it would have just seemed impossible. It seems sort of ludicrous like like this is not the way quantum physics works. And yet it it it turns out that that with some sort of caveats but uh sort of that that’s the the right way of thinking there’s a surprising there’s a a shocking scientific fact there >> and what do you think about this uh for example um we used to have the future humanity institute uh in Oxford and that uh shut down recently and I’ve heard some people talk about like there’s the the need for a more like macro strategy u thinking do you think that there’s kind of a void for that and like um if you if you were to like decide that you know we’re going to try to do something that works on this like imagining the future and let’s say you you got a $100 million or something what what would you do sort of what would be the best course of action do you think >> so uh got $100 million to do what exactly >> to to basically like put this idea into action of like a serious discipline of imagining the future that we need that >> around this specific sort of idea of I mean imagining future objects, future entities. >> Yeah, I could see it. I mean with the macro strategy point maybe those are should be kept separate because there could be just like the imagining is one thing and macro strategies and you know seems actually like something else where you >> interesting I just think about sort of where these ideas have tended to come from and usually they’ve come from people you know who are deeply embedded in particular disciplines fairly early the first arguably the first paper about AI you know is during about sort of 1950ish what I’m getting at is this type of work it tends to be sort of deeply grounded in particular fields but then also to be somewhat anomalous within those within those fields. There’s very few papers. So there’s this sort of str a few strange papers by Richard Fineman and David Deutsch and Richard Yoser and and a few others about sort of essentially proposing a what a quantum computer is. Um and then once you’ve got the model sort of normal science sort of takes over and other people with very similar training develop the idea further but they’re not really they’re not for the most part cons sort of you know conceiving of fundamental new affordances I suppose they they are a little bit well certainly it’s a it’s a strange type of work what I’m getting at is when it’s sort of very distributed and very embedded in sort of a deep understanding of particular parts of the world it’s actually hard to see the the idea that I I still quite like is probably sort of some sort of a vision prize where you just solicit sort of vision papers of this type. God knows how you get judges who are broad enough to actually judge this sort of well and you’d really you wouldn’t be looking for the flashiest sort of outcome or the most flashy sounding sort of possibility. you’d be rather looking for sort of depth and surprise somehow which is the thing if I think actually good example I’ve given things like topological quantum computers as as an example of something that’s surprising an even more surprising thing I think is something like public key cryptography which just sounds impossible you shouldn’t you should have to exchange key material to be able to communicate privately and yet it turns out that that if you understand sort of one-way functions and related ideas sufficiently well no actually it’s not required so that just has there’s a sense of shock there. This ought to be your first thought when you hear about public key cryptography is I can prove that’s impossible and then followed by oh and here’s how it works. Um it’s just it’s just amazing. Um um actually there’s an interesting connection to to nanotechnology. Ralph Merkel I guess is one of the inventors of public key cryptography and he’s also one of the pioneers of of modern nanotech. That’s interesting that he did both those things. I mean because they’re very different at some level. interesting thing to that he’s done both. Maybe I’m maybe I’m maybe I’m empirically observing that I was wrong before in my statement about the depth of embedding into particular disciplines. Maybe Merkel’s a counter example. >> I don’t know. Well, I think um I think the interdisciplinariness is in general like an interesting point or that’s at least at foresight we often uh we often know that breakthroughs seem to come from interdisciplinary crossover and maybe especially now I think that’s also isn’t that one of the like sort of in AI for science as well. It’s it’s that’s what I’ve heard from like paper QA and these sort of things that they want to be able to sort of translate between the disciplines because there’s you know a lot of um information may not have been translated between and that’s maybe where a lot of like lowhanging fruit could be now for scientific discovery. >> I’m very interested in this what I call literature. They often had a very hard time getting published. Not always, but that seems to be a very common feature. And I think it’s because they’re not normal science in sort of the conventional they’re proposing some imaginary future object and saying it will have these properties um uh sort of more or less well defined. Sometimes it’s very well defined. I think Churing’s paper on programmable computers is very clearly defined. Sometimes it’s very vague. Alan K has this lovely paper about the Dina book which is sort of essentially it’s like a much improved iPad from the 1970s but that’s it’s quite a vague paper. Nobody would know anything about that paper if Kay hadn’t then gone and actually done a whole ton of it. Um um so but the the question of how you create sort of a venue for that and how you make it into a serious like what’s the right standards sort of normatively there uh I suspect a lot of people I mean those papers often have zero impact for a long time uh not always but but surprisingly often so I don’t quite know what to do about that. I mean, maybe that’s part of the point of the, as I say, sort of if you have sort of a vision prize and you solicit sort of ideas, uh, you’re creating a little bit of a sort of a venue for it and some sort of normative standards around it. And they’re not necessarily going to be appropriate for everybody, but but, you know, if they’re appropriate for some people, maybe then they sort of start to set it. It just becomes a venue for conversation about interesting goals that people didn’t formally have. Yeah. I suppose actually, I mean, in some sense, the computer science community does this a little bit. I think about things like the programming language community where a lot of that much of the most interesting work is about finding new fundamental abstractions which can be used to control computers. And so there’s a little bit of of that feeling although even there much of the work that’s done it’s just let’s figure out how to improve our type system a little bit or whatever sort of very technical sort of incremental work very interesting but um I’m just sort of riffing and not really making much progress I’m afraid >> I think that that’s been my um experience also with the the vision paper I think is really interesting but I think what you say is is true on like almost who would be the judge you know you would just need someone with great intellectual taste or something like that. And I feel like the judge is almost going to be harder to find than like >> ideas >> actually Drexler you know you know had a lot of problems you know sort of being I think supervised is my understanding at MIT because you know he’s not in a what’s he doing he’s not he’s inventing a field and it’s interesting the extent to which his adviser which I believe was Marvin Minsky did actually supervise a lot of people who did that in different ways in in sort of in different different fields often seemed to be Minsky I guess co-invented con focal microscope and but then his PhD students here they did things like invented scheme I believe and just all these other things um sort of across a wide range of of different areas maybe that’s sort of an example of almost a factory for these kinds of ideas >> yeah it’s somewhere where it’s okay to be more interdicciplinary or more just like inventing a field apparently you know um to be very exploratory let’s try to talk about the the tools for thought point um cuz I you know you’ve obviously written about this but if some listeners are unfamiliar with this concept do you want to just like explain what what we mean when we say tools for f >> sure I mean the the phrases due to Ken Iverson is his name from the 1960s he’s the inventor of the APL programming language and he got very interested in the idea what was his thesis it was notation as a tool for thought and he was very interested in this idea which has a very very long history that that somehow the symbols we use influence the way in which we think and they influence sort of the expressivity. This is an idea I suppose it’s most often associated with sepia and wolf sometimes regarded as a bit disreputable sort of the idea that maybe our language influences the way we think a lot. I think if um you sort of take a long enough historical view it’s clearly true obviously language was an enormous um bootstrapping event for humankind. of how old it is, we don’t really understand the origins of language very well. And then these uh sort of apparently related things, it’s not quite clear where mathematics comes from or to what extent it’s, you know, is it using the same circuits as language is it, you know, what’s it sort of grounded in? But again, that seems to sort of have expanded our world really dramatically. The ability to build models on paper, it’s such a shocking thing. You think about something like uh the first nuclear weapons, you know, this began as graphite squiggles on tree pulp. It’s such a shock. Zelad in I think it was 1933 really realizing that nuclear weapons would work. Uh and it’s based you know it’s not based on empirics. It’s based on well it’s some empirical input but um you know it’s fundamentally based on models of the world which we you know write down with pencil and paper. Um, and that’s such a wild fact about the world that we do have these these tools for the thought. It seems to be true that I believe it’s true anyway that much of sort of human history has been about gradually upgrading and improving those tools. So, you know, you invent numbers probably just to solve resource allocation problems. You gradually improve your number system. You invent Hindu Arabic numerals um and those enable all sorts of things. And we don’t know necessarily successor states going to be found. You know, it’s very tempting to think that we’ve sort of found the, you know, modern alphabets and modern written language and so on are the the be all and the end all, but almost certainly that’s not going to be the case. You think about sort of neural interfaces and it’s not at all apparent what the right way of being sort of linked mindto mind is, but probably much more interesting ways are possible than merely speaking to one another or smiling at one another or, you know, using your sort of facial expressions and body language and touch and and so on. But but those are all sort of they’re design questions. So both rooted in, you know, what’s possible in the world, but also something about imagination. A real sort of epiphany for me as a kid growing up was the very first time I ever used a paint program on a computer was Mac Paint on a Macintosh. And it was just shocking to have these tools. I could fill something in. I could, you know, create a hatch effect. I could do these kinds of things. and then sort of learning years later that Bill Atkinson who recently passed away you know he just he’d invented a a lot of those tools there’s a there was a pinball arcade somewhere in Certino where a particular light effect was used and that’s where he got the idea for the marching ants effect that is to some extent still used today to indicate sort of a um you know that you’ve selected a a region um uh these are all kinds of we see these being done in the external world and then start to internalize them. And as you internalize them, they change the way you think about they change the thoughts that you can have. To your point, I don’t have very I wish I could talk better about this. I’m a few years out of data actually. I just haven’t really been thinking that much about it, but um it is very difficult to talk about. >> For some reason, it is. But I recommend people read more because you you have a bunch of posts about it on on your blog. >> Yeah. Will I feel like I’m to some extent asking you the same question because it it almost feels like a hyper entity question again. But like if you think about tools for thought, are there any that you think would be like really high leverage for creating a better future if we were able to like have them and integrate them more broadly or are they >> not really that comes to mind? >> No, >> I suppose I am I mean historically pretty interested in the connections between sort of capitalism and the changed nature of society. So the invention of medium of exchange seems to have you know obviously it’s had a huge impact on the economy but it’s weirdly also had this you know impact on the ways in which people behave. I guess the Protestant work ethic is probably the best known sort of example of of this Max Max Fabers notion but um it seems to have modulated behavior in in lots of other ways. this this I think it’s urban legend no two countries with which have a McDonald’s have ever gone to war with each other or something along those lines which I doubt is true but it might have been true at one point it’s express it’s trying to express some some idea though about the way in which our human behavior and collective behavior isn’t just modulated in the obvious economic ways but is also u modulated in other ways by our economic system and in in some sense certainly I wouldn’t say money isn’t just sort of a tool for thought but that’s part of what it is. Uh I think that has become more and more clear through work on cryptocurrencies over the last few years where a lot of those people sort of they explicitly they’re very interested in the question of how does the medium of exchange we use modulate sort of collective intelligence. How does it enable new types of collective action? So in that sense um I suppose it’s not it’s not an individual tool for thought but it is certainly not so much it certainly is a a collective tool for thought. So I’m you know very interested in that question but I don’t have a good I don’t have a good answer. I I love that that things like protocol labs has you know this series of events on public goods and sort of connections with crypto world. I think that’s incredibly interesting. I also think it’s very interesting you know Vitalik Peteran has this terminology DAC that it’s not very explicitly connected to Ethereum yet as far as I know but I find it very interesting that he’s thinking about ways of in some sense economically modulating sort of our focus on defensive versus offensive technologies. That’s a another potential link, but again, not very concrete. >> That That’s really interesting. Yeah. And it’s it’s the funding the commons, the events. >> Yeah. Funning. That’s right. Yeah. >> Yeah. Um Yeah. No, I think uh I think we’ll leave the tools for fought there. But it’s it’s a really interesting idea that I think we should definitely try to look look closer at and maybe the best questions don’t have any any simple answers as well. I think that I want to like try to wrap a little bow around this conversation because I think the reason I do enjoy reading your post so much is that I never really know what I’m going to get. Uh and that’s quite quite enjoyable. I think it’s because you’re like quite you know like it feels like you’re quite intellectually independent and you you don’t mind being a bit contrarian but not in like not out of the need to be contrarian but more like you don’t mind it if it’s necessary. So it’s this like principles first rather than like you come from an ideology which I find a lot of other people maybe are more like they interpret everything from based on their ideology basically. So my question for you is like is this is this something that you do consciously and like is this something that you work to uphold yet? Do you do you think that this is important? >> I don’t think terribly consciously. I mean there are various specifics about the ways in which it’s enacted which are fairly conscious but the overall point of view I mean I you know trained as a theoretical physicist and I think that’s you know out of just an obsession with understanding sort of underlying principles and then actually while doing that at some point you realize oh you need to get quite good at mathematics and the the probably the most useful thing I ever did for getting better at mathematics was treating books about and papers about mathematics not really as things to read but rather as sort of sets of problems to solve. So you see a theorem and you you don’t read the proof. You try and prove the theorem. Um and that’s a I don’t know I mean thousands and thousands of hours of that. And yeah I mean that was sort of done as an expression of personality but I think at least as much done as just out of a sense of oh this is the most useful way to get better at this and to understand it more deeply and then probably sets up a whole lot of sort of habits of mind that then carry over later. I I will say I mean I I’ve often noticed yeah I’ll be sort of writing something writing a paper or essay and there’ll be a section that I absolutely hate and that’s often a sign of like a real opportunity. Typically what’s going on is I’m sort of regurgitating the standard story and realizing that there’s something a little bit wrong here and you sort of tweak and tweak and find what what is wrong and then sometimes it sort of it will all unravel and even when you go back and sort of explain the actual point of view that you’re arrive at. It might not actually be very different than the conventional story. Often there’s a lot of wisdom in the the conventional story somehow. It’s yeah you’ve really thought it through and uh sort of understood it in a way that that you didn’t previously. But that’s I don’t know it’s a compulsion. I’m not sure it’s particularly admirable or chosen or or anything. It’s a >> more inevitable. >> It does seem good just in the sense that like there there are obviously like a lot of I think interesting intellectual communities >> like I think the progress movement recently or like the effect of altruism movement or um these sort of things. And how how do you think because it seems good in order to avoid group think which I think like a lot of uh these types of communities are you sort of end up at risk uh of >> of this? Do you do you have any um recommendations like on how to like disagree politely or something like that? >> Um disagree politely. I wish I was better at it. It’s funny actually how much of disagreeing politely is down to individuals. Um, so I don’t know. I think somebody like Toby Ord, you know, who’s a founding sort of one of the founders of effective altruism, or Patrick Collison, who’s one of the founders of the progress studies, or Alexander Burch, you know, who’s a a key EA person. They’re so nice and sort of polite themselves. Um, you can’t really disagree in a nasty way with them. I mean, you would feel ridiculous. It’d be like being mean to, I don’t know, the Dalai Lama or something. Yeah, maybe that’s a a a good way of describing it actually. You want to be polite in conversation have opponents who are themselves sort of exceptionally polite people in many ways to to sort of to disagree with. I don’t always understand some people I I will find myself provoked and it will seem to me that they’re lovely people and yet there’s something in me that is responding and I find it difficult sometimes to to sort of to disagree in a productive way with those those people. Um, I think that’s usually more a indication of something unresolved in me than it is something about them. Yeah, it’s a good question. Something something I remember seeing years ago was I don’t even know how you say her name, Camille Puglia, being interviewed and she was incredibly interesting and she had so many fascinating things to say, but at a few points she got on the subject of academic English departments and she had a lot to say about this because she’d spent much of her career in in them. And I remember sort of thinking after a while, I wish she like I wish she’d had better enemies just in the sense of like I got the sense that a lot of the people who she’d found herself opposed to there didn’t deserve somebody of her sort of level of imagination and and whatnot. I mean it was a minor point. It it wasn’t obviously the main thing about her her life, but but I remember sort of thinking, oh, you want to pick good enemies to sort of do interesting work. I don’t mean enemies in a particularly serious way there, but but sort of people who you disagree with in very productive ways is uh somehow very helpful. See, it’s a lovely thing I think about about effective altruists in general. They’re just a delightful group of people to disagree with. Such an interesting group in that that reg. >> Yeah. Well, maybe that’s the way we put the bow around it is like we you need better enemies, but also like maybe better friends to disagree with or just >> I think I mean that’s really the right way of putting it. It’s uh I I’m certainly not referring to them as as enemies, but that that was it was actually was the the puglia seeing that that interview really had a actually a very deep impact on me and and and sort of realizing that you do need it’s very healthy somehow to be disagreeing with uh in sort of in in very productive way. Yeah, I actually I mean this is a bit self advertising maybe about foresight, but I do think foresight is a really interesting community in that sense that there’s not there will be very very many different viewpoints at a foresight event has been been my experience which definitely I think just yeah opens up your range of uh like what what’s possible I think or like what’s possible and yeah >> also it often feels like a sort of a crossroads in a really interesting way where there’s sort of caravans of set out from many lands to come there and uh sort of meeting and and exchanging views. >> Yeah, >> it’s very interesting in that kind of way as a almost a facilitator. It’s almost like a I suppose an Amsterdam coffee house in the 16th century, something like that. A place to to bring people together in productive so actually so hopefully they can have productive disagreements and whatnot. >> Well, maybe that’s the note that we end on. Thank you so much, Michael.