The Network As A Program With Nate Foster

TITLE: The Network as a Program with Nate Foster CHANNEL: Jane Street DATE: 2026-06-01 ---TRANSCRIPT--- Welcome to Signals and Threads, in-depth conversations about every layer of the tech stack from Jane Street. I’m Ron Minsky. It’s my great pleasure to introduce Nate Foster. Nate Foster is both an old friend and an accomplished researcher at the intersection of programming languages and networking. He’s both a professor at EPFL and a visiting researcher here at Jane Street where he spends a day a week mostly focusing on work with our networking team. So, thanks for joining me.

Thanks. I’m a big fan of this podcast. It’s great to be here. To start out, I’d love to hear a little bit more about your origin story. Like, how did you actually get into this whole computer science world in the first place and also into the kind of place that you’ve gotten over time in in this area of research that you think about? Yeah. So, I I didn’t actually get into computer science really until partway through college. Uh I started out as a physics major and I like to say I was a failed physics major. Um I was uh uh you know taking the sort of standard sequence. you take some mechanics, some uh electricity and magnetism, you do quantum. And as I was going through that sequence, I I found I was kind of liking physics a little bit less. And on the side, I was taking CS classes. Um and I just loved them. They were I was kind of fully engaged. I was, you know, doing well. And I really liked how all the pieces kind of fit together. you know, I could understand after a couple years like how a a program, you know, gets implemented down through a compiler, down through uh the ISA, through the chip all the way down to to gates or even lower. Um, and that was very pleasing. And I also really liked uh the idea that, you know, unlike physics say there’s a lot of creativity a lot of these abstractions that we have in computing are really designed by people. So things like lambda calculus and functional programming like who would think of that as a basis for computation and yet it runs on a on a computer. I just found that really beautiful. Yeah, it’s kind of an amazing thing about computer science is like it’s not like a science in a traditional sense or at least not totally like a science and that you’re not studying the natural world. You’re studying human creations and yet those human creations are very bounded by both all sorts of things about like the physical world of what you can reasonably implement and bounded in important ways by mathematics. Right? It’s something kind of amazing about the field. Um so I can say a bit about uh about how I got into research. Um that was uh a bit of an accident. I think many things in my career were sort of an accident. Um I I had the idea that I might want to go to grad school, but I was maybe a little too shy to articulate that. Um and there was an ad uh when I was my junior year, an ad for a summer research project working on a Java compiler. Um and uh I thought, well, I might want to, you know, get a masters or maybe even a PhD. I’ll I’ll spend a summer doing research. Um and my mentor was a guy named Kim Bruce who’s a a programming language researcher. um and spent the summer hacking on that Java compiler, working on type systems and just fell in love. Actually, what were you doing in Java? What were you trying to do with the type system? Yeah, so this is way back uh Java originally was sort of a big advance. You know, it sort of brought things like garbage collection to the mainstream for the for the first time. I mean, garbage collection had existed, but you know, in a in a language that was being used really widely, that was that was new, right? And and just to be clear, shockingly, 40 years after the emission of garbage collection, that’s right. garbage collection existed, you know, decades before, but it’s like the single biggest advance in like programming language productivity and something like 40 years between the time of invention and the time of mainstream use, which I’ve always found like a really shocking fact. Yep. And Java also had this type system. It had a static believed to be, you know, sound or mostly sound type system. Um, and that was very exciting to the academic programming languages crowd because they, you know, we we love type systems. Um and yet again the mainstream languages at the time you know C and C++ don’t really have sound static type systems. Um so the research we were actually doing was um adding polymorphism to Java. Uh now Java has generics but at the time it didn’t and there was a process by which uh the community was allowed to sort of propose you know ways of bringing generics to Java. So uh my adviser Kim Bruce had had such a proposal and you know uh he’s at a small teaching college so he didn’t have grad students so he found undergrads to work on implementing this type system. So is the one that you worked on the one that was in the end adopted? Uh no so the one that was adopted uh was a very nice design by uh Phil Wadler and Martin Odriski um called GJ was the name of the original paper um and that’s that’s the one that eventually made it through the community process but there were actually lots of teams working on uh different proposals. Uh there was one out of MIT from Barbara Liskoff and Andrew Meyers, one out of Rice by Corky Cartwright. Uh Kim Bruce had his um and they all varied in their notation and how expressive they were and uh you know different different features. Um but Oh, that’s super cool. And I I don’t think I’ve ever heard of another language process that quite worked like that. Yeah, I don’t actually I was too young to know the reason why why Sun Microsystems who who was sort of controlling job at the time why they did it through this community process. But um perhaps the feeling was you know type systems are kind of this thing that seems very simple but actually can be quite subtle. You know it’s very easy to design a type system that has a soundness bug or has some maybe you know runtime cost. So they tried to leverage leverage the smarts of the community to help them design it. Okay. So that’s how you got your first taste of programming languages. What happened then? So uh then after undergrad I spent a couple years in in England uh doing uh not computer science actually but I knew I want to come back and do a PhD um and I ended up at Penn uh where I uh worked with Benjamin Pierce who’s a friend of yours as well. Um and you know I thought that I would work on I don’t know more type systems or maybe something in semantics you know the kind of really uh macho kind of big topics in programming languages. And when I got to pen, Benjamin said, “I’m working on this data synchronizer. That might be a good project for you to start with.” Oh, and Unison. Yeah. So, Benjamin uh has this uh tool built in O’Camel called Unison. Uh it’s a it’s a file synchronizer. Um and at the time he was trying to make it uh generic so that you could synchronize data that was in different formats. Um and the project was called Harmony. Um so I was I’ve never told this story publicly. I was actually a little disappointed because I thought, you know, I’m going to work on a really crazy type system or some, you know, new fancy denotational model and instead I was working on a data synchronizer which sounded like a kind of systems problem. Yeah. But embedded in this project um was this really beautiful uh programming languages problem. Um and uh Benjamin and his postto at the time Alan Schmidt had discovered that uh in order to synchronize data in different formats uh you need to convert between those formats in in two directions. Um so say you’re synchronizing maybe a you know a version of your calendar in uh the standard you know ICAL format and maybe another version at the time XML was all the rage so you know in XML. Um, so it has somehow the same information, but but there’s some differences and uh if you’re going to synchronize them, you you need to basically convert from one into the other and vice versa. And so they’ve been writing these conversions, you know, writing sort of functions that go from, you know, version A to version B and from version B to version A. And very quickly realized these two functions that were writing separately, they’re like really close to each other. They’re they’re almost inverses. And as programming languages researchers they said well maybe we should have some kind of abstractions for writing these these two mappings. Um and that became what’s called lenses. So the idea was to have you know one abstraction from which you can derive both of these functions right and actually you worked on this lens idea early on and in fact has like had a whole big life afterwards. There’s all sorts of people in like the Haskell community who have all sorts of variations on the lens idea and use it kind of all over the place and seem to be very excited about it. Yeah. So lenses are one of these ideas that I think was sort of in the air. We we Benjamin and Allen were not the first to discover it. Um there was prior work in the database community. There was prior work actually in the functional programming community. Um someone named Lambert Merittens had worked on a very similar idea. Um, but their definition sort of was particularly sort of clean and and and elegant and so it it kind of caught people’s attention. Um, and yeah, I worked on it for my PhD. We worked on a variety of uh of lenses. Um, playing with, you know, what we could what we could express, you know, how how complicated could we make the functions um using types to make sure that the lenses had good properties. Um, for example, you generally want to know that, you know, if you’re making changes to one side and then you use a lens to push the changes to the other side, somehow those changes should be reflected accurately, then you can try to characterize that using some kind of some kind of laws or some kind of uh specification. Um, and then, you know, building this out and trying to trying to figure out how this could turn into useful useful tools. Um, so I yeah, spent six years working on lenses. Um, had a lot of fun. Uh, and you mentioned sort of that lenses have kind of caught fire. uh o other people much smarter than us uh you know took the idea and sort of generalized it. So in Haskell um there are versions of lenses that are um not exactly our definition. Uh they’ve sort of loosened some of the some of the mathematical definitions but in ways that allow them to have lenses for all kinds of different structures and it’s really cool. Okay. So that but that was still you doing like kind of a little bit systemsy but still like what feels like pretty kind of straight ahead PL research. Y but that’s not what you do today. So yeah, how did that happen? So I have this funny kind of uh right turn in my career. Um I as I was finished my PhD, I decided to apply for faculty jobs, got hired at Cornell where I spent 15 years on the faculty. Um and then uh I decided to take a kind of a gap year um really for personal reasons. My wife was finishing grad school. I wanted to stay near her. But also I think although programming languages and functional programming is is my home, I kind of felt like, you know, I’ve thought about lenses for six years. is I’ve kind of had all my good ideas. I want to have something else to work on. And so I decided to take a leap and uh and go to Princeton um where I was I worked with Dave Walker and Jen Rexford on problems in networking. And I described this as a leap of faith for me. It was really a leap of faith I think for for for both sides. Um Jen in particular kind of hired me almost blind. um and uh thought that it might be you know fun to have a project at the intersection of programming languages and networking and and hired me as a postto you know with really no background in networking um and and that was the start of my uh of my current career push. So maybe this is a a good moment to stop and like just talk about what is actually your current research like what’s the overall thrust of your research? What are the kind of core ideas that you’re trying to explore there? Yeah. So I I’ve spent really the last 15 years working at the intersection of programming languages and networking. Maybe before I answer my you know my my own goals, it’s worth giving a little bit of context. Um starting around you know 15 years ago um there was a big change that happened in the networking community. um it’s become known as softwaredefined networking and it was driven by lots of factors some kind of economic some based on uh changes in hardware but the real technical changes were really twofold. Um some of the big organizations like like you know tech companies, cloud companies, they wanted to have more freedom to change how their networks work and that was very difficult for them uh with how networks were built say circa you know 2005 to

2010. And the second was their networks were getting really really big. You know they’re building these huge data centers and managing the complexity of all the different protocols and control algorithms and such uh was getting out of hand and so they wanted a different way to do this. Um and so uh the sort of ambient you know background thing that happened is uh a bunch of people uh both in industry and in academia proposed sort of a new model for networks and it was based on the idea that you could really specify the behavior of a network somehow in software. So you could write programs or uh or somehow generate you know the behavior of the network uh from from these higher level descriptions. Um so to me as a programming language researcher it’s kind of obvious that you know there’s many ways you could sort of do this. You could have maybe databases of of you know network uh structure that you somehow push down into into the infrastructure, but wouldn’t it be better to have a language a language that um lets you maybe operate at a a higher level than what the hardware lets you do? So you can sort of design, you know, different data structures, different algorithms, and write those in ways that that are more natural for humans. Um you might want to be able to uh compose programs. You might have like a network that’s doing multiple things and you like to be able to think about each of these things separately and then somehow compose them together. Conversely, you might have a network that’s doing multiple things and you you don’t want them to interact. So you want to keep them separate. And so being able to express, you know, different forms of composition or isolation using language abstractions is very natural. And then third uh to tame this to to sort of manage this complexity and this operational challenges you know having tools that could understand the semantics of what the network is doing and then kind of turn the crank and understand okay here are all the behaviors of the network and some of them maybe are good and some of them might be bad uh you know being able to verify networks or or reason about networks was was a third thing.

So just to make sure I I I’m getting this right. So in some sense the situation X anti was there were a bunch of switch manufacturers right there was a kind of pre-existing notion that was kind of built out of like the early internet days of like what does a switch do and what is IP and what are the protocols and then there were actually programming languages of a kind. There were like configuration languages that you could use to choose the behavior of an individual switch. And then the kind of classic way of managing a big network is you like figure out what the layout of this physically should be, how you want to wire things together. You buy a bunch of these devices, you wire them up, and then you hire a bunch of network engineers whose job is to kind of very carefully configure all of this stuff so that it doesn’t break the network and you get the properties out of it that you want. And then SDN in some sense is like no no no we’re going to like I guess you still have to do a lot of the physical layer work but then the configuration part is very different now instead of configur configuring each individual piece you try and have an overall program that tells you how the network as a whole works. That’s right. And then like the advantage is I guess one advantage you’re saying is much more configurable so that you can like get more behaviors out of the hardware than you could have gotten out of like the stock thing that you would get from the vendor before. So there’s like a kind of faster cycle to be able to iterate on new things that you want to do. And then maybe like another big thing is the ability to reason about the thing that you’re doing at a higher level. All the things you said about composition. Like the thing I hear from that is like oh I can actually like predict how the thing is going to behave and understand based on a relatively small program what the behavior of this like sprawling network is going to be and make sure that properties that I care about are enforced. Yep. And I think like one thing though that’s different though like you should we should understand that you know what counts as a program that you might want to run in a network is going to be different than the kinds of programs that you might run on a server. So there still are some things about you know about hardware for example due to the speeds and and you know the sort of scale that networks operate at. You’re not going to write an algorithm that sort of you know has a heap and is sort of you know allocating memory on every packet at every hop or something. That’s that’s going to be crazy. Um so there you are in some of a special domain. Um there’s also um you know some differences uh you know just from the fact that networks are part of the infrastructure. So even in say a single organization there’s usually a desire to know things about um how how different pieces of the network are are isolated from each other or again even in a single organization you may have multiple units that are responsible for uh controlling the network and so you you still have in some form you know federated or distributed control. Um so this is what makes it you know not just let’s take lambda calculus or let’s take Java and you know that’s that’s our way of writing network algorithms. there’s still uh some interesting domain specific structure um that needs to be explored. Yeah. And in a lot of ways this echoes to me the story around hardware synthesis, right? Where again it’s like there’s a big graph structured computation thing and you need some kind of language for expressing it and then there’s a lot of play there of like how sophisticated is the language and how much does how much power does the language give you to kind of reason about the thing you’re doing and to flexibly compose bigger designs out of smaller designs. Um, and but again like there are these very profound limitations on what you can do in a hardware design. Like you’re not going to write the same kind of code at all. I guess it’s a question of like how much of traditional PL theory applies in this world, right? Because it’s like the constraints are very different. Yeah. So maybe I can tell you about a couple of projects or we can work through them. But um, one thing that’s been very kind of exciting and kind of cool for me is um, despite all these differences, there are some things from classical PO theory that that match up quite well. Um so one of the languages that we discovered early on is a framework we call netcat. Um and this is a language that we’ve designed for describing not the whole network. So we’re not describing the behavior of the end host. We’re not describing TCP and congestion control. We’re not even describing what’s called the control plane which is the sort of brains of the network that decides you know which routes you’re going to use and how to respond to failures. So we’re just describing really the you know how the network processes packets at the at the forwarding level. Um and we’ve been working on a sort of series of DSLs for for several years in in kind of following on from my postto at Princeton and we had you know sort of I would say ideas about how to make this these languages you know what features we might need how to do it in an elegant way. Um, but there was this exciting moment when we realized that all of these languages lined up really well with a system called CAT, which is shortand for cleaning algebra with tests. Um, which had been around for for a couple decades. In fact, uh, Dexter Ken at Cornell had had discovered this this framework and had worked on it for a long time. So, CAT is really it’s a pretty high level mathematical abstract framework um, that’s meant to be uh, kind of a model of of kind of standard imperative programming. um we can talk more about it but um that actually lines up really well with at least how you know you can think of the forwarding behavior of of packets through a network. And so the reason I say this is exciting is you know although you start from a domain that seems to have all these you know weird primitives and weird constraints you can sort of extract from it this thing that looks a lot like you know finite automa and state machines and all the things you learn about as a as a second year computer science student. Um and and that alignment is is quite cool to to to discover. Yeah, I sort of think of this as like the one weird trick of programming language theory, which is that like there’s this like complicated and messy and very human process of writing programs. And then it turns out a lot of the best ideas in programming languages come from relating the thing you’re doing to very simple mathematical models. Uh, and there’s something nice about languages that have this tight relationship. Not all of them do, right? Right? There are languages that are kind of much messier mathematically, but but the ones that have a kind of tighter and simpler mathematical foundation, at least my sense is tend to be better at generalizing. Like the features that you add to like solve one problem turn out to solve lots of different problems and compose nicely with other ideas. You can sort of having this like foundation gives you this kind of nice playground where the different ideas that you come that you can come up with end up integrating better with each other because they kind of all fit into this relatively simple worldview. Yeah, I completely agree and actually if in our academic paper on netcat that’s exactly what we said in the introduction, we sort of said the value of this framework is not so much that we can do something we couldn’t do before but we’ve now aligned ourselves with this you know with this theory that is actually backed by you know going all the way back to to clean to like the you know the 1950s. Um and that gives us both kind of some confidence that what we’re doing is is maybe right or or at least you know sensible. Um, it also gives us a whole bunch of uh constructions and tools that we can pull from from formal language theory. And in our work, we’ve actually used a bunch of those tools to build compilers, to build verification tools. But especially, it’s what you said. So, as we’ve extended netcat with new features, you know, now we’re kind of back into being, you know, confused researchers just playing with examples and trying to make things work. But having this structure of of, you know, cleaning algebra with tests has provided guidance to us. So for example, we we’ve worked on a probabilistic version of this language which is very useful in networking because you have unexpected things. You have you know traffic and you don’t know how much traffic there’s going to be. You have failures. You don’t know when the failures are going to happen and you may also have like randomized algorithms that are used to load balance across the network. Um and so reasoning about the behavior of all these things requires you know reasoning about probabilities. So we extended out with with you know probabilities. Um worked out the semantics. Um, and that was very, you know, not obvious and and and a bit subtle. And I think if we didn’t have the structure of of cat and, you know, and and other other associated theories, um, we would have very easily ended up with a language that was kind of incoherent. We’ve been talking about a mix of like some about like what softwaredefined networks is and kind of in the broader sense and then also about your research. Like how would you explain like the particular take that you and the other people that you work with have on on this field? Yeah, I mean I think the the main kind of slogan is the network should just be thought of as like another program and that’s a short sentence that maybe sounds very trivial. Um but it’s to networking people it’s a really different way of thinking about things. Um you know for for many years uh innovation in networks has been driven either by things that happen at the hardware level. you know, a vendor comes along with uh a new router that has a hardware pipeline that has some extra feature and then that turns into you know something new that you do in load balancing or congestion control or something or queuing or something else or you know driven by you know standards bodies for a long time. You know, if you want to do something new, you have to uh have a good problem, have a solution, and then go convince a vendor to implement it. Go convince a bunch of other users of networks that, you know, we should standardize this, get it ratified by the by the standards bodies. Um, it’s a very long, slow process. And so thinking of networks just as programs is is like, well, just as we don’t go ask permission from, you know, Intel or or Nvidia when we want to deploy a new algorithm on hardware, we just write a new program. we should have the same freedom um with networks. How much of this is is basically gated on the sort of existence of the hyperscalers of like huge companies like Google and Amazon and stuff that have enormous networks they they configure that they can configure this way like in some senses an issue around the domain of administrative control like if I need to primarily build a thing that interacts with things I don’t have any control over then I can sort of see how I’m gated by the standards bodies because I have to get everyone to agree to like share the language with which we communicate. Whereas if I have my own enormous network that I’m going to configure, I can just think of the whole thing as a program and then I maybe have to think about I maybe have to think about the standards bodies on the edges, but within the network I get to do that. Yeah, I think that’s right. I mean if you again go back and sort of tell the the you know intellectual history of software defined networking it very much did emerge when these really large private networks uh became a thing and the companies that were building them wanted to have the freedom to you know basically define new features at you know software time scales um and it’s true the the thing I was just sort of you know throwing shade at of you know vendors and and standards bodies well that’s what built the internet and the internet worked because you took, you know, tens of thousands of autonomous systems, tens of thousands of, you know, networks built by different organizations, different people on different hardware in in the originally like even all the way down to the physical layer. They were using, you know, different ways of moving bits around. Um, and you connect it all and make it all work and interoperate. So that’s, you know, the internet was designed for really, you know, connecting up all these different networks and making it and making it work worldwide. uh and that’s why you know we ended up with certain solutions. But in these really large networks with you know hundreds of thousands or even millions of of of computers and uh a comparable number of of switches and routers um you may also want to you know have the ability to define those networks and to optimize them and to have them implement certain features. Um and so that’s that’s a big part of the story you know more more economic than technical but um I think it’s an important change that you know you do have sort of again even in a large organization you do have often multiple uh teams or multiple units that are involved in this but still there sort of you know one one ultimate unit of control that gets to to define the goal. Do you think the idea of the network as a program generalizes to the open internet? This is one of the big challenges that the community’s been thinking about for decades. Um, so one of the kind of paradoxes of the internet is it’s it’s so successful that you can’t change it. And this is something that the community has been very worried about uh going back quite quite a number of years even at least to the ’90s. Um, so there have been a bunch of efforts to think about well you know the internet works really well for at today’s scale at you know for today’s applications. Um, but there are things that come along that we’d like it to do. And how are you going to change, you know, these tens of thousands of AS’s? How are you gonna how would you, you know, decide to move to a different routing protocol for the internet or, you know, a different way of moving packets around? You you can’t, you know, turn it off and turn it back on tomorrow with a with a big flag day. Um and so there’s this there was this sense and you go back and read sort of papers from 20 years ago, people would talk about oification, the idea that the internet structure and its kind of skeleton were kind of setting in and it was impossible to change. So there’s a whole other uh community that’s been thinking about how we could design an internet that is extensible and evolvable. Um and that’s that’s a sort of very rich cool space. Um and there are people with with with cool ideas. Uh some of them involve program languages but a lot of them also involve you know different architectures um uh different ways of getting extensibility. And to what degree do you think this idea of your network as a program like has caught on has been influential has kind of changed how people build networks in practice like both among like there’s like an academic audience for this but then also lots of practitioners and companies and also all of these hardware vendors like like how how has this idea kind of propagated over time? Yeah, it’s funny, you know, if you if you track sort of ideas or or uh these these these trends, um I mean my view is this is this idea has become, you know, just the way things are done. In fact, I can back this up with a little bit of evidence. a few years ago um some of my collaborators including uh Jen Rexford and and Nick McCuin and and some others we we wrote a paper um uh basically arguing that you know the network as a program was here and and it was a sort of vision paper for a short conference and it got rejected and we were a little bit miffed. I mean we get papers rejected all the time but you know we kind of we’re really proud of this paper. Um but the reviews actually said you’re describing the way the world works like your ide your ideas aren’t spicy enough like this is this is how things work. So I think to some extent you know the network as program or software defined networks just is how things work. Now if you for folks who are familiar with the sort of original articulation of software defined networking of course there are some ideas that 15 years ago people were saying well you know we should we should build you know centralized algorithms or logically centralized algorithms and that idea has not so much caught on in in in practice or um another example is you know there was a big push I was played a small part in it in um making uh network routers sort of truly programmable almost every piece of their functionality could be specified in a program and again for mostly economic reasons that idea has not caught on. Um but at the same time you know the the sort of ability to change the the say hardware pipelines of these routers is is is coming just not in sort of the way that it was originally articulated. Um so again I’m I’m biased but but I I think it really is the way the networks are going. Um, and you know, the path the path towards this vision of like you sort of can can write code and and get the rich behaviors you want. It’s not exactly smooth in the way that was predicted at all times, but but it’s the general trend is in that direction. So the the SDN idea is interesting to me because I feel like it it falls along another theme that’s like a little different from the way you framed it. Um, where you’re kind of talking about this basic, you know, being able to specify things in cleaner ways, getting better abstractions out of it. And I think that’s all part of part of it. But another thing that I think has been very valuable in this kind of work is just adopting the culture of software and the kinds of tools of software. Like there’s all these domains in computer science that just have picked up different approaches and techniques for building things. And if you look at the way in which people think about management of databases or doing hardware synthesis or networks or building traditional software, like the techniques are actually all really different. And there’s a bunch of really good ideas that have come up in software that I think aren’t as clearly expressed in the other domains and things around things like the way that you do like version control and code review and testing and things like that. And in like the old world of networking where you know you just go and configure the switches to do the thing that you want, you kind of don’t have this centralized place where you can do all of these pieces. And so sort of in some ways separate from the kind of like nice semantic improvements which I think are like super important. This thing of like allowing the kinds of tools that people use and the kind of engineering approaches that people use in software to apply that to domains like networking seems like another advantage of of this whole of this whole thing which I guess is like maybe like hard to summarize nicely in an academic paper but I feel like in practice is a big part of where the advantage comes from. Yeah, I completely agree. And um the other so if we’re talking about sort of trends that got you know a bit of maybe hype and a lot of attention and maybe the hype wasn’t quite deserved or you know things didn’t quite play out. Um the this area I think well you’re talking about something more broad than just formal reasoning. I think you’re just talking about adopting kind of modern software practices to you know keeping things in databases or repositories having uh not you know humans don’t just log into a router and and sort of you know yolo a change they they uh run it through a process maybe there’s some even some checking um absolutely um and you can actually I mean you can go back uh people were thinking about this even at the ISPs in the 90s you know they were large ISPs were sort of the equivalent of the hyperscalers back then they operated sort big networks that were complicated, expected to work. Um, and they had started to experiment with some of these ideas of, you know, having at least sort of centralized, you know, specification of the of the functionality at at companies like AT&T and then how to realize that. Um, but the other piece is uh verification. Um, and this has I think is also it’s maybe not as not quite as far along, but it’s um something that’s becoming quite common place. All the hyperscalers are doing it. Uh, there’s also some startups. There’s uh there’s of course many academics who are interested in this idea. And here it’s that you know well if you have some representation maybe it’s not a beautiful representation in netcat but you at least have some program that describes how the network is supposed to behave or how it is configured you could start to apply you know all the tools of software engineering um testing tools validation tools even verification tools um and this could then become part of your workflow. So, you know, before someone decides to push down a change that might change the routes between two data centers, you could check, you know, is this going to break connectivity anywhere else in my network. Um, and that’s that’s a a good idea. Yeah. And I guess you actually hear lots of stories of like large companies managing large networks where config changes break things and cause huge outages. It’s actually like one of the biggest problems you run into is people having a config change that unexpectedly has some semantic behavior that they didn’t expect. Yeah. I mean I think to be clear this area is really not done and it’s in part because although uh this I think I’ll take network verification you know it sort of took this one layer that became exposed. So the idea that there’s a a centralized either database or or program that is defining the behavior of the network and then that gets pushed down to the routers who then you know realize it that that gives you a place where you can sort of interpose and you know you can intercept snapshots of the network and start to start to test them or reason about them. And so that’s that’s what people mostly have done but networks are much more than that. Um and they’re you know they’re distributed systems. So they have all the complexity of distributed systems where you can have failures that you didn’t expect and uh interactions that weren’t part of your model. And so we’re definitely not done. I mean you still see outages due to human error or you know flaws in the model all the time. Um and I think this is you know this will get better but it is really hard to reason about these these complex systems that have components you didn’t even really think about or know about. uh you know multiple control loops, funny interactions. It’s a it’s a it’s a true puzzle that um you know requires some new ideas to make progress on. Do you have a good example of a kind of problem in the space that is like now pretty well solved? Like a thing that people would pretty routinely get wrong in the past and that now there are there are at least in some places good verification checks to help people not make those mistakes. Yes. Um, I’m going to twist the question slightly and not say that people got it wrong, but that there was also um sort of conservatism like people were afraid to make changes because they weren’t sure what the impact of those changes would be. Um, so this is by the way like a huge part of the network engineering story as I’ve experienced it. Like I think part of the job of a good network engineer is tell you know, right? Yeah. It’s like that’s too complicated. We’re not going to do that. Yeah. One example that’s I think it’s no longer research. it’s it’s sort of been fully reduced to practice is reasoning about these snapshots of you know the so-called forwarding plane of a network. So of course networks have changes happening all the time. There’s failures, there’s different controllers that are maybe you know monitoring the system and making changes but you can pretend that there’s a snapshot and you can you know a consistent snapshot maybe that that you can uh sort of extract and then reason about and uh that snapshot can be modeled using tools like model checkers or SAT solvers. these automated theorem provers that understand first order logic or custom tools like netcat is such a tool you could put it into netcat and then ask questions about the model. So this is something that is pretty widely done and you know mostly just works. I mean it it does what it’s supposed to do. Um and so uh it you know it does catch certain errors um in the sense that you know if you you could write a specification like you know these two hosts should always be connected no matter what routes are being used and I want these two hosts to be able to to send traffic to each other or these two hosts should be isolated and if something goes wrong and somehow there’s a path between them that’s bad. Um so these kinds of properties you can check um and you know if if your control plane okay the brains of the network makes a change that would violate that property you get some kind of signal or exception um so and that’s you know I think that’s uh that’s useful in say say cloud companies um but uh what it doesn’t solve of course is uh what do you do you know so if if you’re getting a failure of a property because the control plane has done something bad Um then what? Um so it’s not like we’ve we’ve sort of made networks perfectly reliable or perfectly able to satisfy their specifications. Well, although if you if you like if the controller wants to do something and you know it’s bad, can’t you just like I don’t know like in a software context be like oh we won’t merge that PR, right? That’s like you can if you can catch it at the time where the change is proposed before it’s accepted then then you can like there’s something you can do about it which is just again like the software like the network engineer you can say no. Yes. Although uh there are cases where that may be sort of the wrong move you know if it has uh some other effect like you made this change because of a failure. Do you you know do you keep the failure unsolved because you decided to reject this change that violated your uh your spec. This is where things get a little murky. Oh interesting. So like maybe basically there’s a control system sitting on top and you know merely saying I reject your change if the control system is not going to then do something better um you may not have actually improved life. Got it. No that makes sense. So a lot of the things that you’re talking about seem like they involve a pretty rich connection between a bunch of ideas that you can develop in an academic context and then a bunch of industrial use cases. Some of these at like places like the hypers scale or some of them at the at the actual network switch vendors. And I know that you spent some of your career with various kinds of engagements with the kind of industrial side. Can you say a little bit more about like how that worked and kind of how you’ve integrated that into into your kind of career and research and approach to thinking about the space? Yeah, so this is something I actually love about the networking research community and I say the research community not the academic community because it truly involves you know the hyperscalers and the switch vendors. um somehow you know this the particular community that you know identifies as researchers and networking uh is not just driven by universities and PhD students. It it involves all these all these different entities. Um and so it has a really nice mix of you know you have people doing pure theory but you also have people who have you know designed the the you know wide area backbone for a giant cloud company. Um and they’re all coming together to share ideas. So that’s really cool as a researcher because you have this, you know, relatively small group of people who all know each other and they’re working on related ideas and you therefore have this kind of, you know, quick you can have quick transitions of research ideas getting into getting into practice. Um, I think you know my my sort of original home community of programming languages also has this. Um, but as we already said the time scales are often much longer. We talked about garbage collection from you know the 1950s till the 1990s or you know type systems similar similar many decades um and actually you know I mean Jane Street’s a very uh prominent uh company in in the functional programming space so uh people people understand that you know functional programming is being used industrially um but Jere’s a little not unique but you know there aren’t there’s not as much of a conversation between sort of mainstream programming languages as used by you know the millions of developers in the world um and you know the academic community. So, it’s something I really love about about about the the networking research community. Although maybe there’s like more now like Rust is another example of a language where there’s been a lot of very rich connection between academic and industrial. Yeah. Yeah. I I do feel like it’s it’s it’s been changing and uh and yeah, similar kind of short time scales to cool academic idea, you know, appearing in some mainstream language and then be being broadly used. Um there’s another piece of this that’s kind of interesting which is um when these ideas like software defined networking first came out uh a lot of the companies decided deliberately to sort of engage a broader you know ecosystem a broader community um uh quite famously sort of you know Google was sort of very interested in ideas uh like openflow which was the early SDN sort of standard um but chose to do it in open source for you know strategic reasons um but that sort of created an opportunity to build a community around these And were you involved in thinking through and helping set any of those standards? Uh no, I was uh not involved in OpenFlow at all. That was sort of that was already pretty baked by the time I did my posttock with uh with Jen and David Princeton. Um I did get involved in this uh second phase of uh of trying to design uh languages and associated hardware for describing the behavior of individual routers, switches, network interface cards. Um we worked on a language called P4. Um and that was a similar sort of community effort. Um right and like OpenFlow was more like you get to set the routing table like a little bit more general than that but like it was like there are like tables that you get to configure there and then like P4 was more like you kind of get to write the whole switch. Yeah. Uh Openflow people like to bash on because it was kind of cartoony. Uh I think its designers did not intend it to be a cartoon but you know there’s sort of a big gap between OpenFlow’s model of how a router works which is basically there’s one big lookup table and you’re going to cram all your logic into this one big lookup table and the reality of you know high-speed routers and switches which have pretty complex pipelines with specialized units that do certain things. So I think the original hope was that somehow openflow would be realized by you know smart compilers teams you know people who would sort of lower it down to these pipelines that’s actually a pretty hard task and so P4 had the benefit of being sort of a second mover it was sort of a you know a second attempt and it just exposes the structure of the pipeline um so you do get to customize what happens but there are certain things that can be just exposed in the language or in the programs itself right and and the work with P4 this actually involves some pretty deep engagement on the industry side for you as well Yeah. So I chose during my first sobatical uh at Cornell to to go be a part of um be a part of the company that was developing one of these programmable switches called barefoot networks and then also the P4 community. Um this was a choice because I think I felt some uh maybe I don’t know if imposter syndrome is the right right word but I felt very much like I was you know I was sort of the programming languages academic who was sort of going and you know cosplaying in in networking and I wanted to understand at a deep level you know how how does how does a router really work um and so going to a hardware company seemed like a great way to do that and then how like I’m a little curious like both how did feel making that transition like I feel like going to be a a CS academic is like a choice right you can go and do computer science in academic context you can go and do it in industrial context and I’m kind of curious like why you in the end like made that move and what you felt like you got out of it and how it affected kind of your thinking and research after that. Yeah. Uh I mean I can tell you the the kind of personal history. There’s again you know mentors who uh who sort of provided advice. I I was very unsure. Um uh my colleague Fred Schneider who who you know well at Cornell your adviser. Yeah. So I remember talking to Fred you know should I do this? Um and he was like absolutely you know if you have a chance to go kind of deepen your uh knowledge and expertise in a space that’s going to pay dividends down the road. I had other mentors George Fargazi who’s at uh UCLA now. He was sort of like you know go go do this. And then Nick Mchuan, who was the co-founder of this of the company and one of the STN pioneers, um he really sort of, you know, opened the door to me to to come be at the company. Um so it did it did not feel like a big risk. Um I uh I confess of course, you know, really not knowing anything about hardware except for my uh VLSI class from from sophomore year. Uh it was and not that I became a hardware designer, but like the hardware industry is amazing. I mean you have you have just these people who are you know the best at what they do at designing circuits at optimizing them at you know physical layout at uh integrating all the pieces from different vendors going to the fab and getting it manufactured. I mean it’s just amazing what what is involved in making a chip and the startup had some some real veterans and people who really knew what they were doing. It was um some some alumni of of Texas Instruments and then others from around the Bay Area. Um and I I learned a lot. It was really fun to to be with those kinds of experts and you know learn how how hardware works and how it’s how it’s built. Do you have any like concrete examples of ways in which your research after was different than your research before for having done the experience? Well, there’s a line of work that came out of that of this I think I wouldn’t have done. I can’t take credit for it. Um my colleague at Barefoot Chang Kim he’s now at Google and and does lots of AI infrastructure for them. um had this idea that if we have routers that are programmable like fully I should say on the router chip that barefoot was designing literally almost the entire behavior end to end you could specify in the program there were a few things that were fixed but pretty much you know you sort of receive bits and then you can write a program that parses those bits into some data structures and then you can write some code that you know interacts with different memories and you can change the bits and you can run them through different hash functions and other functions and then you can, you know, spit them out the other end. The whole thing you could really specify. And so he sort of realized, you know, this is just another kind of processor. It’s a processor that looks a lot different than a CPU, but uh it uh it has a little bit of memory, a little bit of state, um and it’s very high throughput. And you know, if you think about a data center, what would it mean to have a bunch of processors? You know, if you take a typical path in a data center, you go through, I don’t know, you know, three to six different hops, maybe. what if you could do a little bit of processing uh on on those uh on each processor. Um and so we worked on a number of applications of this idea. It later became known as sort of in network computing. But the the question was like what could you do like what functions could you kind of cram onto one of these weird quirky resource constraint things? And then I think the deeper question which was quite controversial is like what should you do? Should should you do anything in the network? And if so which functions would make sense? Um, and this is controversial for people who don’t remember their undergrad networking class. You know, there’s again from the internet design and there’s a sort of famous paper people often read that argued that you really don’t want to put functions and services in the network uh that uh will only benefit you know certain users because then everyone sort of pays the cost. Um, and so there’s there’s sort of some orthodoxy around you know keep the network very simple, right? This is like the endto-end principle or something. Yeah. So that’s an example I I you know really enjoyed working on those those uh those projects and this idea is still very controversial you know whether we should have in network computing but I also believe that eventually you know some form of this will become inevitable and we see this a little bit you know some of the networks that are supporting ML fabrics have you know more than just sort of traditional packet forwarding um and there are some uh services that uh that were developed in some of these systems um things like you you know, consensus protocols or ability to make observations about what’s happening in the network fabric or, you know, things like caching certain certain data that I think really are good ideas or could be good ideas, you know, in the right context. So, that’s something I definitely wouldn’t have done if I didn’t understand hardware. Yeah. And that’s I think something that we also do some amount of of like finding places where there are like critical bits that you want to accelerate that you want to kind of put at particular choke points in the network. sometimes for monitoring reasons, sometimes for like filtering and shaping of traffic. And I guess in our context it in some sense helps that there’s a there are like a small number of kinds of flows that come up all over the place and like multiccast is maybe like the biggest one of like we you know it’s the way in which exchanges distribute market data. Uh there’s actually a lot of complexity around the fact that like multiccast is not super well supported by uh by all the hyperscalers. this thing that I remember when I was eons ago in grad school and learned about networking like multiccast was this important thing that was going to be the way that we delivered video to people and like that turned out not to happen. Um in fact I think there was like a basic confusion it wasn’t clear till later which is that like it turns out multiccast which is like mechanism for essentially broadcasting data which involves laying out trees that you can like use for the automatic transmission and like take advantage of the network switches capability of like copying data in parallel down multiple paths concurrently. Um and it seems like a great way to get the same data to lots of different people. But I think the thing that wasn’t clear at the time was that the data plane was going to be super cheap and the control plane was going to be really expensive. Meaning you would have a huge amount of bandwidth for sending data around and actually very little space for like the control data with which you would like lay down the the trees. And if like lots of people want to consume lots of different data and sort of these different logically separate multiccasts, then like you just weren’t going to have space to kind of specify all that. But in our world, there’s a relatively small number of channels, a small number of things we want to kind of get to everyone. And so the old multiccast idea kind of works in this context, even though it kind of totally failed in the outside world. And now it’s just like, you know, in the cloud there’s just like no multiccast. Like in fact, sometimes the cloud vendors will give you things that look like multiccast, but they’re implemented really really badly and slowly. And it turns out those are there for if you have some ancient application that you want to run and it thinks it wants multiccast then you can like give it that interface and we will do a thing that like kind of delivers the right packets at totally the wrong time scales but you can get some like legacy software to work that otherwise wouldn’t work at all. Yeah, I think it’s well I want to talk about multiccast some more, but um I think it’s also interesting, you know, you sort of mentioned a way that uh your your grad school, you know, version of networking and certain distributed systems kind of was wrong. And I’ve described a few cases where I worked on things that kind of, you know, didn’t end up being as successful as we as we hoped. And I think that’s actually really healthy. I mean certainly university researchers should be working on things that don’t work out you know and I don’t mean don’t work out like you couldn’t solve the problem you couldn’t prove the theorem you couldn’t build the system or whatever but like don’t end up being the way the world works I mean that’s you know part of being in a creative innovative community is like people are trying wild things and you know not all of them are going to be the right thing to do or you know the the the good thing to do at a particular juncture and I think when communities get too conservative and you know you you’re supposed to do things the sort of orthodox way. That’s actually a recipe for stagnation and that’s a little bit of the the oification that happened in the internet community was like there were these principles which are good principles. You know the end principle is a good principle but you know you should know when to break it and and you know it may not be the right breaking it may not be the right thing for every scenario but if we never are allowed to go revisit that rule you know somehow the world just got a little smaller. Um, so I I love communities that, you know, in some unruly way are like, you know, advancing and making progress towards towards greater and greater things, but along the perimeter there’s just all kinds of chaos and and people doing crazy things. Um, I know you watched this video, we we just put on a conference called Nines, um, which is a new conference for the networking community devoted to this idea of like let’s explore new ideas and we’re going to be, you know, not we’re going to be wrong, but like we’re going to be very accepting of of of new ideas. that’s going to be the main criterion we use to evaluate papers. And to accompany some great papers that that got published, we also invited some sort of, you know, luminaries in the field uh to to lend credibility to our effort and also to um to tell stories about, you know, about their experience with new ideas. And there’s a video from Scott Shanker, what I got wrong about QoS. So Scott is like, for those who don’t know Scott, he’s one of the giants in computer networking. He’s another, well, Scott’s not a failed physicist. He actually is a successful physicist but he who switched to networking. Um but he uh he he was you know involved in in bringing uh networks that have uh support for quality of service uh in the you know late 90s and you know put his name behind it. Wrote lots of papers and he gave this very thoughtful piece about why things didn’t work out and you know what what was wrong but I mean also what was right. Um but I I just love that piece. I think academics should a be doing wild things and then b you know not be so shy about reflecting on you know hey this thing didn’t end up catching fire or or or being you know the way the world works but but it was still interesting to explore right yeah I I think we’ve had our own kind of evolution in in in within Jane Street where I think early on in some sense we only did things that worked like you know it was a small company and there was lots of lowhanging fruit and lots of opportunities and we kind of started out with like a working business and like you know there were lots of small things you could do to make that business better and and almost always make things better in relatively short time frames and we still do a lot of that. There’s a lot of plucking of lowhanging fruit but as the organization has grown we’ve had more opportunities to try bigger things and to do projects that sometimes take years to bear fruit. Um and there’s something lovely about that as well. And I think AC academia like can and should and does you know take an even more extreme version of that where like you can take more bets that you know know some of these bets might not work out for 10 or 20 years and like that’s okay and I think is is an you know it’s an exciting way of moving forward like the bounds of knowledge and to be able to do that in a way that isn’t as you know constrained by the needs to like get like the next practical thing up and running. So you’ve had this kind of industry experience at Barefoot. Now you’re here at Jane Street with a different kind of industry experience. Like I was sort of involved in the story of how you got here. Like we’ve known each other for a long time. I think I first met you uh when you were I think a PhD student at Penn in Benjamin Pierce’s office talking about the lens work. Uh so we’ve known each other for a long time. But I’m kind of curious like just from your perspective like like you know you spent a long time knowing kind of vaguely of Jane Street as like this weird trading firm that uses functional programming languages and I’m kind of curious how that process felt to you of like what you thought about James Street like in the past and like what in the end led you to think ah actually maybe coming and spending some time here and doing work here would be an interesting thing to do. Yeah, I think you know we we had been talking I was sort of I mean Jane Street for those who don’t know is sort of quite prominent in the academic functional programming community which I consider myself a part of. So you know Jane Street sometimes publishes papers at at ICFP which is the the main conference in functional programming uh sends people to the conference. I was sort of aware of, you know, here’s this company that is, uh, using a functional language for much of its work. Happens to be my favorite functional language. Um, and also, you know, really takes on hard technical problems and is willing to sort of make these longerterm investments, uh, in, you know, systems and tools. And so I was sort of aware of that. Um, and I think, you know, we’d been talking, I’d even had some meetings with some folks in the networking team about um, some of the efforts that were being made to to sort of, I would say, bring SDN-ish ideas to Jane Street’s network. um I wasn’t around but my understanding is you know James freeze network has has sort of you know emerged uh from being a sort of small sort of smaller you know sort of human- managed network to something that’s well maybe not quite as big as the hyperscalers getting big enough that you want to use those same ideas of kind of you know top down specification having some tools to understand what’s going on um being able to really make changes with with confidence so you know uh to me uh the chance to kind of work on those kinds of problems uh at a place that has uh and we should talk more about the the firm’s culture, but uh we talked about, you know, sort of ideas that fail. Um J has a very open culture. There’s a a wiki page, you know, you can just do things. And I I think that really appealed to me as well, the idea that, you know, we’re not going to be sort of just doing things, you know, sort of a conventional way. We’re going to um think about, you know, cool maybe new ways to solve uh to solve these problems um and then get smart people to work on it together. The other thing that um I don’t know if I’ve told you but it was definitely in the back of my mind is um although the research community has this lovely sort of tight embrace with um the the hardware vendors and the the cloud companies and now the ML companies um I was very excited about the idea that well maybe financial networks are different right there’s certain things like multiccast like latency and latency at a different time scale than what the cloud companies care about and you know whenever you just take a problem and you tweak it a little bit. You add some new assumption or different constraint. Often that leads you to a very different kind of solution. And so I thought it might be fun to understand, you know, some of the unique problems that that finance and and Jane Street has and then to be at a place where, you know, there’s there’s going to be the the smart people and and resources to go solve some of those problems. Yeah. And for all that academia is a place where you can like work on all sorts of wild ideas. It’s also the case that in lots of contexts academic work gets kind of caught up by the industry thing of the moment. And I think in networking like the hyperscalers have that shape of like it’s a legitimately big and important problem. And like you see that like networking papers kind of overwhelmingly want to think about that. There’s like an older fun example of this in the garbage collection world where there’s like a stretch of years where all the garbage collection papers are about Java garbage collection and like Java was like a very particular kind of language with a very particular approach to garbage collection that like skewed the way things were done. Like one interesting aspect of this is the way in which you tune a garbage collector. Like you always have to tune a garbage collector trade-off between space and time of like how much time are you going to spend collecting and versus how big you’re going to let the heat go. And then like the traditional way of doing this in Java collectors is a kind of roofline model where you’re just going to be like how much memory can I use? Well, how much memory do you have? Right? And I you sort of say it’s like sort of you know it’s like you have this big hulking enterprise application and it will run on a box and it will be able to use every bit of RAM on that box. And like then like when it gets close to using it up then it’ll have to like work work harder to collect memory and that’s not like the only and like for many contexts not really the best way of tuning it. In fact Okamel has a very different way of tuning its garbage collector which operates in percentage terms and like a whole different set of heruristics and stuff is in mind but like for a long time kind of all the papers were structured around this roofline model and then like eventually it breaks out of that. Um and so anyway, I sort of kind of am open to this idea that like yeah, there’s like it’s often useful to kind of break away from the standard thing that everybody is doing. Yeah. So what now now you’ve been here for a while and worked on some interesting problems here like what are examples of problems that you have seen that come up in the gene street context that actually do look different from the problems you see in the outside? And one example we wrote a a short paper on this um just getting at this question of multiccast we’ve already talked about a little bit. Um so uh I don’t want to summarize the paper but um the basic story is uh you know support for multiccast hasn’t gone away but it’s really been sort of diminishing at the hardware level and although you know I think most if not all trading firms of course use multiccast because that’s what the exchanges are are giving us in terms of data um the uh the trend has been that commodity routers have gotten uh they have more features uh they have a lot more bandwidth but they’re getting a little bit slower. So the latency is kind of creeping up as you make more complex pipelines that can do more more processing of every packet. And then sort of relative to the growth in things like bandwidth, um support for multiccast has sort of been flat or even getting a little worse. Um and so we wrote this paper that was just asking you know this this is not yet I wouldn’t say it’s like a a looming problem. Um but if you sort of you know follow the trends out for some years it could become an issue. And and then in the second part of the paper we sort of asked well what are some really different designs that we could think of? Um are there different kinds of uh fabrics that we could build? Maybe things based on you know optical networks or circuit switching. Um and there’s interesting trade-offs. You know you can you can actually build a fabric that sort of delivers lots of traffic you know simultaneously to to lots of places. Um but then you have sort of a proliferation of of traffic everywhere and you have to filter it. Um so you have you you sort of have this trade-off between yeah sort of easy cheap delivery with certain kinds of networks versus the ability to inspect classify and then you know split and drop. Um and so that’s a paper that I think you know sort of uniquely could be written in in this context. Do you think there are lessons to be learned by like from for from from places that are you know in the kind of in more of the hyperscaler mode from the things that you see in more trading style networks? I’ve I’ve looked over time at like the kind of designs that people have built for doing all sorts of you know kind of standard web style problems of like you know an example that I remember talking to some of the engineers there about is like the way in which Twitter does distribution and analysis and transformation of the sequence of tweets which is like you know now on the modern scale a pretty small data problem but at some point it was a bigger one and I remember looking at that and thought multiccast would be really useful here like And I wonder to what degree whether the kind of magic powers of multiccast are kind of under underappreciated and underused in other kinds of context and that like maybe they should pick up them maybe they should pick them up more than they actually do. Another another thing that shows up a lot in trading context and show up a lot and showed up a lot in my own PhD is state machine replication. um which is a kind of core idea for building distributed systems and like shows up a ton everywhere and like you know cloud providers are and are also building things with this. Um but multiccast is like a super nice primitive for building efficient state machine replication systems and it’s not one that seems to show up a lot in practice and that’s like another just concrete example where I suspect it could be more useful. I think it’s, you know, multiccast gets used for lots of things in a trading context, right? There’s using it like in a very kind of local environment in the middle of building a certain kind of more or less supercomputer, right? That you can have like lots of systems that are hooked up to each other with multiccast and it’s a way of like giving you an efficient bus for just distributing messages to everyone. And then it can also be used as a way of connecting data across different organizations and like that’s the exchange side of this, right? they they deliver multiccast as a way of efficiently and fairly getting their data out to all the many people who are consuming it. And then that same multiccast tree kind of extends into the network um of the consumer and maybe that latter one is like very trading specific right the kind of cross cross institution version of it but the inside of the company version of it or the inside of a system it feels to me dramatically underused. Again, I don’t have your distributed systems instinct, so I can’t I can’t kind of spar with you on that. But I one thing that I’ve been pondering recently is um and this is inspired by a paper by Nick Mchuan and his student Sundar um that they published just last year. You know, most network infrastructure is still based on, you know, the the good old packet packet switching model. And there’s reasons that we moved to packet switching in the in the 60s. Um you know, it gives us uh efficiencies. we don’t have to schedule things. We don’t have to understand uh reserve capacities and so on. It’s a it’s a very you know simple building block that has really nice properties. Um but uh and you might wonder like what does this have to do with packet switching? Well the sorry with with multiccast um the challenge with packet switched uh routers is that building support for multiccast is actually pretty complicated. So doing that with low latency and you know finding building in sort of the heart of a of a router, you know, uh a unit that can move packets along, you know, any combination of input and output ports um at speed, you know, maybe maybe even doing some some queuing at the same time. That that’s pretty complicated, I guess. And it’s complicated because like if all you had was the, you know, a single multiccast tree to distribute along, then like packets would come in and you’d copy them out to multiple outputs and everything would be cool. But you actually have many different things happening concurrently. So you both want the parallelism of being able to like emit out of multiple wires at the same time, but also you have to tolerate all this dynamism. Yeah, there’s like some fundamental tension there of like you can’t like at the physical layer do things completely in parallel if some of the resources you’re trying to address are busy doing something else and and this part of a router is sort of the you know the middle part that has to run you know the fastest. It’s what determines sort of the rest of the performance of of your whole router. So that’s generally complicated. So what this paper that Nick and Sundar wrote is they were looking at machine learning workloads uh in particular training workloads which are many and often very regular you know and so why are we doing packet switching at all? Why don’t we just uh understand you know what’s going to happen when and then take those schedules you know this this data here is going to be delivered according to this permutation and this one according to this permutation and then you can build a much simpler switch that just understands how to implement these you know permutations on a schedule and you know Nick has deep hardware understanding so he you know the paper explains sort of why why this would lead to you know simpler cheaper uh faster switches um but for that use case it it seems like you know the right the right way to do things if you were to be able to boil the ocean and build all the infrastructure from scratch. So the to me the sort of intellectually interesting question is you know we have certain kinds of networks that can do these uh tasks that are otherwise expensive or slow or hard. We know how to build networks that can do those tasks very well. Um, but we’re sort of afraid to build them because, you know, there’s so many benefits of of of packet switching that we can just sort of spray these packets into the network and whatever resources we have will be used efficiently by the by the herd. May maybe in the future we’ll start to think about going back to some hybrids where we do a little bit of both. Um, yeah. And and in some sense we are kind of boiling the ocean or maybe like making several new oceans or something, right? is like the whole ML world is creating this enormous revolution in networking and you have both like much higher demands in terms of throughput and latency just because of the in fact in part because of this very regular process right a lot of the synchronization in machine learning training is this kind of barrier synchronization where like you know you have a bunch of hardware in parallel doing a thing that hardware is actually very deterministic and so it finishes pretty much at the same time across multiple and then they need to like exchange their tensors really quickly and all that time where they’re exchanging tensors. I mean you can do some overlapping but like sometimes you can’t do overlapping and any uncovered like communication is just time where these very expensive GPUs are just idle and it’s just wasted money and so there’s a huge amount of pressure on these networks and people are also increasing the heterogeneity of the networks because now you have like the networks on the inside of the like I guess Nvidia uses maybe everyone uses this kind of somewhat odd terminology of scale up versus scale out where like scale up is like the really fast little network and then scale out is the work beyond that. Um, so you that could be a context where you have the freedom to go and try like very different things. Yeah, it’s actually I mean in networking it’s a pretty exciting time because people are playing with maybe not ideas quite so as radical as like building just a scheduled crossbar uh in in you know making that be the building block but there is a lot of innovation in uh transport protocols, collectives, co-optimizing you know the low-level CUDA code and the communication code. Um so things feel very like suddenly like oh we can we can sort of play with all these pieces of the of the design. Um and then you know because training and serving AI models is sort of the you know the central problem of the day for for systems you kind of get immediate uh feedback and when something works you know people get very excited. So another thing you’ve been working on while you’re here is BGP. Maybe you could like say a few words about what BGP is and then talk about like the problems that we’ve run into that you’re working on making better. Yeah. So maybe one of the areas where you know Jane Street was sort of living sort of in the past. Um we have a now by now big and it’s it’s been growing a lot um worldwide network that connects all of our sites and um although we have a lot of tooling and analysis it was still we were still expressing what we wanted the wide area network to do in terms of configurations for individual BGP routers. Um and that very much feels like like sort of the dark ages. Um so maybe I’ll I’ll quickly explain what BGP is. BGP is um was originally designed as the routing protocol for the internet. So you have the internet with all these tens of thousands of autonomous systems. You know every organization is its own system gets to decide uh how it routes traffic to other autonomous systems and you need some protocol that these so-called AS’s can use to uh agree on how traffic flows. What’s an AS? Autonomous system. Oh okay. Right. Um and uh the way the BGP works is um essentially every AS uh knows who its neighbors are and it selectively shares uh information about uh certain paths it knows to reach certain destinations. So for example um our routers that connect to our peers on the internet you know might say well hey we’re Jane Street if you want to reach any Jane Street IP address come to us and then uh those routers will send to their neighbors you know a similar advertisement you know if you want to reach Jane Street I can reach them in in one hop and you can also share other characteristics about about the path. Um so this is sort of the basics of BGP. It’s a so-called uh path vector protocol. It’s disseminating information through the internet about paths that reach certain destinations. And what makes it very rich is there are many uh so-called attributes that you can add to these advertisements. So you can decorate an advertisement not just with I know how to reach Jane Street, but I know how to reach Jane Street with this cost. Uh you can add, you know, sort of tags. You can add a whole bunch of information. And now when a router receives this advertisement, it can sort of compare maybe a whole bunch of advertisements it has all for reaching Jane Street on different paths and it can then make a selection and decide which ones it thinks is best. So it lets every node kind of make a local choice and express its own preferences. Um, but it also kind of quickly disseminates information about all the pass through the internet. And and in some sense it’s sort of like the opposite of what we were describing in this kind of you know I step back and write a big program that lays out a mostly static graph. This is like instead I have like a rich distributed system of individual nodes nodes sharing information and then making local decisions about how to route data. Although hopefully somewhere in there there’s like some reason to think that those local decisions actually lead to good global outcomes. That’s right. What I’ve described is basically how BGP works on the on the internet and um uh it’s actually it was not known for a long time why BGP seems to work so well. You you would think that a bunch of nodes that are that would seem like a very important thing to know making independent decisions. Well, you know, in fact, uh the internet routes are are fairly stable. So, so things sort of converge to uh I mean the internet’s always in motion of course but um if we can sort of pretend that we could sort of stop the world you know the internet sort of converges to the paths that are uh sort of you know at least a local optimum um pretty well and uh there’s a really nice paper by Jen Rexford my postto adviser um and Lixing that explains sort of why this is the case and it turns out that the internet sort of has a kind of structure that comes from the economic relationships. Um, you know, you have sort of ISPs and customers and because the kinds of BGP choices that different players in this ecosystem tend to make, it turns out they’re sort of latent, you know, properties that cause BGP to behave particularly well. Uh, these are the so-called GO Rexford conditions. This is like ancient stuff, but it’s kind of cool that this like unruly distributed system actually kind of works pretty well for for these reasons. Like why did this present problems for us? So uh one of the thing I haven’t said is that BGP is often also used inside of organizations and it’s a little confusing because it was again originally designed for you know the internet where the nodes that are participating are an entire organization like Jane Street or an entire university like NYU but of course inside of Jane Street there are also you know many thousands of routers and they need to understand how to reach certain destinations both internal and external and so there are other protocols that have used in the past but uh for for many decades now it’s been really common to use BGP also internally uh to share knowledge about what paths exist is part of the reason for that basically the dynamism that you need like if nothing else links can fail and you need to be able to recover from link failure yeah I think it’s it’s a really expressive protocol it’s got sort of all these ways that you can sort of cram in information about different routes and make choices and selectively disseminate information it’s widely supported by vendors all the network engineers know it because you know it’s it’s been this way for a long time. Um so you know why not? It’s a good it’s a good tool for you know disseminating information about about the network topology and its paths, right? I guess in in an in an alternate universe where you’re like totally down the SDN route, you could imagine that you could just like look at your overall network and just like decide what you want to lay out. Um and then you don’t have to think about like the communication part of it, but then the d it doesn’t have a that’s there’s no story there for a dynamic. Yeah, I think there’s there’s one of the sort of maybe flaws of the original SCN conception is although you might want to think about your specifications or your your program as kind of being sort of truly one program. I have sort of one objective for my network and I’m going to sort of, you know, check that into a repository and have people review it and and argue about it and test it. But then the way you realize that there are good reasons to have distributed protocols. They, you know, they detect they detect and respond to changes very quickly. They don’t involve lots of coordination. And so if you can map your high level objectives into a distributed implementation, there are good reasons in a large system like one that spans the whole world to do that, right? Maybe you want to compile down to something, but maybe you don’t want to compile down to a static graph, right? Um, so the system I’ve worked on here is a system called butane. And really we’re trying to do exactly this. We’re trying to have well we we have uh a sort of higher level uh and uh in our case it actually is centralized. you know, it’s like checked into our repositories and there’s, you know, if you want to make a change, you go uh propose a change and it gets reviewed just like our other software, but then it gets compiled into uh snippets of BGP, one for every router in the network. And then the behavior of the whole thing is somehow the distributed uh behavior of all of these routers uh exchanging BGP messages with each other and then ultimately arriving at some graph that forwards packets through the network. So I guess the top level of butane is some kind of specification of like what you want the behavior to be and then that compiles down to like the actual concrete configs that land everywhere. That’s right. So what can you say in that top level spec? So the top level spec um I guess you know one thing that that we care a lot about is latency. So um we to a first approximation you know would like certain kinds of traffic to absolutely take the fastest paths. Um, and then you know there’s other traffic that we just want it to get to its destination somehow. Mhm. Um, and we may not actually care that much, you know, if it if it loops around the world to get there, if it takes, you know, twice as long or three times as long as it has to. As long as it gets there, that’s okay. And so this the butane’s policy abstractions are really designed to support uh, you know, sort of the default case just sort of happens. You don’t have to specify very much. and then where you want to say no this traffic should take a fast path um you you can do that and then there’s some other pieces that are kind of a little bit inside baseball but um you know we internally have certain structures to the network and so there’s um the the policy abstraction sort of uh uh expose some of those structures things like you know where are their sites um we have certain expectations about how traffic may flow or not flow between uh certain sites uh there’s a whole set of um ways that we sort of classify and differentiate traffic. So, there’s some features for doing that as well. And then like what’s what is the technically hard part of this story? Like I I could, you know, I could like have a I could I could like write a program that writes a bunch of BGP configs, but I I don’t have much of a lock between like what I wanted to happen and then what’s like the dynamic behavior of it? Like is that like the central problem here? So, let me not answer the question. I’m not going to tell you about the hard part. I want to tell you why why has butane been valuable or what what have we found is sort of the most important parts of butane and to a first approximation it’s a little bit something we talked about before it’s just bringing a sort of software mind scale to thinking about the wide area network so instead of you know operators having some change they want to do network engineers might want to you know move this traffic over to this other path and then go you know make a bunch of changes expressed at the PGP config level to you know several routers. Now you modify a little bit of butane config. The compiler generates the uh the the actual BGB code and then it gets validated. We have some tools for uh visualizing and validating the changes and then it gets pushed to the network. Um so there’s actually not I would say giant technical challenges. This is all sort of fairly well understood stuff, but you know for Jane Street moving us from a world where we’re making changes to individual routers at the BGP config level to being able to work with these other abstractions has been pretty exciting. And I’ll say I was actually not sure what the network engineers would think, but um so far uh people seem to really to really like it. Um the uh ability to kind of take these bigger steps has been very exciting. And then the other piece which maybe surprised me a little bit is um all of the tooling that we’ve built, especially tooling for testing and visualizing. That’s what they’re actually so excited about. You know, the idea that I can like make a change and then we have this UI where you can see what’s the expected you know change in latency say between sites. Um and this is something that you know maybe they could have worked out on paper or could have pushed and then tested. But now we have a model again based on some you know historical measurements and we have a semantics for for both butane and BGP. So we can sort of you know compute what’s the change going to be. We can do this both for sort of like you know propose one change and then and then see what might happen and uh we’re in the midst of making this more powerful. So you can do sort of what if kinds of things like or like what if we lose a link? What well what if we lose n links or you know I’d really like this hotspot to disappear. what’s the you know propose to me a set of changes you could make that would you know move traffic off here while minimizing other changes. Um so that’s a a different kind of I don’t know edit or you know UI that you might like to have where you’re um not specifying a different you know butane policy abstraction you’re really kind of expressing uh constraints on what you’d like the system to get to. So do you essentially need a kind of solver which does this like exploration of the space of other configs? The second thing I described is still is still research. So got it. Uh yes. So we’re we’re actively working on that. Um but yeah, we’re it’s going to look very much like um you know ideas from program synthesis or uh or some kind of solver where you can uh take take these constraints and then explore the space of of programs that might meet them. This is something that you know again maybe to a permanent languages person doesn’t sound all that wild but to do this to the network infrastructure I think is is pretty wild. I mean and and what keeps it safe is that um we we do have this we have a semantics for both butane and BGP um it’s it’s you know extensively tested uh both kind of mathematically and against the actual hardware you know you have to make sure that the way that BGP is realized by the vendors doesn’t somehow differentiate from the from the uh internet standards um and so we have quite a lot of confidence that our model of BGP is good and therefore we can do these analyses and then uh and give answers to engineers before At the end of the day, there’s a there’s a kind of formal methods piece where it is trying to do something that like gives a like provable up to the fact that you don’t know if the underlying model is quite right because the vendors might be doing something differently but like gives you a kind of provable guarantee of the of the fact it is analytically telling you about the network. Is that right? Yeah, I think you know this use of form methods actually may become a lot you know much more commonplace. So I think form methods people often think you know I I build a tool to verify something to like stop bugs. I’m gonna, you know, this is gonna be the seat belt of my of my complex system. But here, of course, we care about, you know, stopping bugs and and preventing us from making mistakes. But the real power is, you know, now we can start to explore. So, we can start to take bigger steps um and even automate some of the exploration of those steps. And that would be unthinkable if, you know, you have to reason about the impact of all these changes, you know, networkwide and let alone, you know, on latency, on congestion. Um and so having models that are backed by some kind of um mechanical uh implementation um that you believe at least you know closely corresponds to what’s going to happen um that’s what empowers these kinds of uh more uh these tools that take bigger steps right and rather than it giving you a single notion of correctness of like I’ve written a spec and does it follow the spec it’s almost like a kind of observability of like I get to like explore different possible setups and think about how they perform and what their behaviors are so that I can like more with more confidence make trade-offs between different design decisions and figure out how I want to structure the network. So one I mean one one surprising use of this that um uh is is just happened in the last couple weeks is uh some of the folks in the team are starting to use it for not actually routing but capacity planning. So deciding you know what future link should we buy. Um and this is something where there is actually a well understood mathematical theory. You can model all the demands for bandwidth and the current network and then figure out how to augment it, you know, subject to what what fiber is available. Um but to do that connected to our current butane policies and our historical workloads and latencies um is kind of cool. Um, so to put it into not just how do we route today’s traffic, but how do we figure out how to expand the network so that we can do a better job of routing tomorrow’s traffic. And I guess that’s just all on the back of essentially having a more complete model of the network that includes these kind of dynamic behaviors. Yeah, I’ll maybe say one more thing. You asked about sort of what was hard about butane and you know we benefited from a lot of academic work. um people like Zack Tatlock at Udub had uh written down a formal semantics for BGP in the rock proof assistant and we didn’t use their implementation but their paper did a really nice job of spelling out you know here’s how BGP uh configurations should be understood um there was also work on designing higher level abstractions for BGP and some of these were really different you know sort of the the analog of like okamel instead of x86 um and we actually chose not to adopt those so our butane policy abstractions are fairly simple. Um, and kind of like Okamel, they compile fairly straightforwardly into BGP configs for different vendors. And we made it, we made that choice early. And uh, you know, I was, if I’m honest, a little bit disappointed. It would be fun to do something kind of more wild, you know, to have a something like like Okamel um, as an analogy. Um but uh I think you know it was the right choice to actually pick something where the abstraction abstracts but in a way that you can when when needed sort of peel back the abstraction and understand how it might map to all the components of a BGP config. Um and and so you know there there are hard problems we could have solved like how do you compile some very expressive policy language onto a bunch of distributed router configs. we did not solve that problem or we didn’t solve it in the in the in the fanciest way we could have. Um maybe in the future, you know, we will start to explore richer policy abstractions, but um in this context at least, I I I very much now agree it was the right choice to sort of pick something relatively simple uh and then you know iterate based on that. Yeah, there’s something powerful about having an abstraction that has that isn’t just simple in terms of its semantics, but is simple in terms of like how it elaborates like how it goes from the highle thing that you want to the actual thing you’re running. If you think about people who are trying to like in close detail engineer how a system behaves in all sorts of different aspects just giving them that kind of vision onto the behavior. And I think the point about Okamel, it’s sort of both. It’s both like a fancy wild highle language and also like it has a relatively straight ahead like compilation story that where it’s relatively easy to understand from looking at the ochamel code how that code is going to execute and so that you know that’s I mean there are trade-offs here obviously I think more optimization is good but also more straight ahead compilation makes it easier to think about what’s happening y how did the process of taking like these ideas that that that you came in part from you and in part from the team and like turning this into like a thing we could actually like roll out into the network and use. So it’s it’s kind of amazing it’s worked this way. Maybe I could say a bit about kind of how my engagement as a visiting researcher has gone. So you know I’ve been spending about a day a week here for a few years. I’ve spent some uh periods of time where I’ve spent you know more intensely you know a whole week or or you know several days in a row. Um and so a lot of the uh design of the system was done by you know sitting side by side with some network engineers uh some folks on our uh our team that that builds networking tools um and you know trying to understand our problems trying to come up with some solutions and then uh and then prototyping them and then uh you know seeking feedback from others on the team and and continuing. So for me this is um been really fun because uh you know in my in my day job as a professor I’m teaching and and and doing research with a team of students. You know really the the goal is to train students here. I get to work with really great engineers. Um and you know that’s uh that’s a great privilege. It’s not like I’m not my students are also great but you know it’s it’s fun to work with sort of really smart software engineers who can solve problems really quickly. Um, for me it’s always a little bit sad like when the end of my day a weekend, you know, we’ll have had a team meeting and maybe I’ll have done a little bit of development and and synced with the team and then I know that by next Thursday when my when I teleport back in, you know, amazing things will have happened. Um, but but that’s very much been my experience. So I’m sort of, you know, I get the privilege of being sort of a small player in this in this team and then I get to work with uh some really fantastic engineers. has the process of like actually putting it into production been like relatively straightforward or complicated like what’s that how’s that played out? So I think I have had less of a role to play here than in the design of the system. Um again I’ve had the privilege of working with uh some a program manager and uh some other people from the more operational side that have really helped roll it out. Um and there’s all kinds of problems that have had to be solved. We we had butane you know running you know sort of it its current set of features were more or less done and then we did a bunch of testing and then we did more testing in the lab and then we started to roll it out and the first roll out was on um a new piece of infrastructure that James Street had stood up uh somewhere else in the country but wasn’t yet using. So we had sort of a a living lab that we could roll it out and sort of you know kick the tires and see how it worked. And then, you know, rolling it out uh more broadly across the firm was was yet another uh step. Um and you know, this involves uh doing firmware upgrades to the whole fleet of routers, making sure everything’s on a good version. That’s very hard to do. Um building all the tooling for automation so that these things can go into the standard workflows that we have for uh making changes to the infrastructure. So again, I I did almost nothing here, but um there’s an amazing set of people and processes. Um, and maybe say one thing that’s kind of not just uh I mean every every large company has these kinds of processes, but um one thing that’s really neat about Jane Street is a lot of these tools are are really homegrown. And so in some cases we’ve gotten to work with the team that’s actually building, you know, the the deployment tool and uh if there’s something that we need that the tool doesn’t have, you know, they can they can build it for us. So yeah, it’s kind of fun for both good and ill like joot has been on its own like you know 25 year long software adventure that has been like pretty different from other places and like some of that had to do with a language choice and some of it has to do with idiosyncrasies of the kind of business that we’re in. Um but like James Street software ecosystem is kind of weird. Um and you know I think there’s a lot of great things about that. I think there are some pain points about that. Um but you certainly get a lot of control all the way through the stack which is really cool. Yeah. So part of the point of the whole visiting researcher program is that it’s a way for us to kind of build good relationships with researchers. And I think part of the value proposition for the researchers is that connection to the kind of work we’re doing internally is a kind of useful way for them to kind of develop ideas and learn about the world in a way that can influence their research beyond our walls. I’m curious to what degree you feel like that has played out and to what degree like you have learned things that affect how you think about research outside of the stuff you do here. I I’ll confess that, you know, part of why I’ve had so much fun uh spending a day a week in industry is I actually like, you know, maybe playing software engineer. So, I really enjoy I love I love my job as a professor. It’s the best job in the world. Um, you know, there are a lot of interruptions. Even if you’re very uh careful with your time, you end up spending a lot of time on teaching and service and working with students is a joy. Takes a lot of time. So, it’s really wonderful to have a day where um you know, my calendar’s blocked off. I have meetings with folks here, but like J’s pretty efficient with meetings. I don’t like my day doesn’t get filled up with, you know, one-on-one Zoom calls or anything and I actually get to sit in front of a a terminal and and you know, and write some code. I have to be very modest about how much I can do in a day, but for me that gives me a lot of joy. Um, in terms of what I’ve brought back, um, some of the technical ideas that we used in butane, um, uh, are things that I’m now working on in my lab. Um, so I didn’t talk too much about this, but um, under the hood, the, uh, semantics for BGP and butane that we, uh, built originally was based on a just sort of a simple simulator, a little operational model. Um but there’s a more powerful mathematical model um that’s been studied for some time in the networking community based on a more algebraic approach. Um so that’s you know that’s that’s become a topic that I’m working on with my students in in my group. Can I say what what’s the upside of the algebraic approach? What how how would it be better than this kind of more operational models? So the I think the upside of the algebraic approach is um the original vision was to sort of give you the building blocks for building policy DSLs. There’s a paper by Tim Griffin uh called meta routing and that expresses this idea. So um the assumption is you know choices that you make about how to route traffic through a network are very specific to a particular organization. Every organization has their own, you know, policies about how they want traffic to flow, how they want to share information with their neighbors and so on. So, you can’t really come up with a one-sizefits-all solution. Um, and even what information you choose to share with your neighbors, you know, by sharing information about latency or bandwidth or trust. Um, and so Tim anticipated that, you know, BGP might be the assembly language of many policy languages that might all look really different. And so how could we design some general building blocks for designing these policy languages? And so the algebraic approach is that you sort of abstract you know what’s happening in BGP. There’s basically information being exchanged with your peers and then there’s choices being made about the information you receive from your peers. you make a selection and you can model those uh abstractly in terms of some kind of what he calls a rooting algebra because he’s British but routing algebra. Um and once you do this you could sort of you know write down here’s what a routing algebra is. Here’s what happens if you take a given routing algebra and you sort of run it in a graph. Here’s what you know here’s the set of paths you’ll get. But also here are constructions you can do on routing algebbras. So I could take two routing algebbras maybe that one that talks about latency and one that talks about bandwidth and I could run them together. I could sort of glue them together and now I get a routing algebra that shares information about both latency and bandwidth and it makes choices in some deterministic way about you know whether it prefers low latency or more bandwidth. Um but I can glue them together glue their preference function together and now I have a more interesting routing protocol. So you could sort of this sort of becomes a factory for building DSLs. And then the really interesting part is so basically more composability in the space of writing policies. That’s right. Okay. Um and then you can also study you know if I it’s it’s not the case that every instance of BGP or every routing algebra is going to converge to a unique solution. uh sometimes so BGP in general can have this property that um you end up oscillating between sure multiple solutions and you know A prefers this these paths you use those paths for a while but then B is unhappy so then you switch to the other paths and then A’s unhappy so that’s bad you would like that not to be the case and you can study generally you know what conditions on my algebra do I have to have to ensure convergence got it anyways these are these are kind of old ideas um but we started to play with these um in my lab and I think you know the the the grander vision is um we’d like to take the sort of original vision of SDN that you can write a sort of top level program but we’d like to have the imple the distributed implementation based on BGP which is widely supported and such and so this this could be sort of the IR of that kind of system cool so like a kind of richer compiler can you imagine that work eventually reflecting back into the kind of things that we’re doing here uh potentially um I think we as I mentioned we were sort of somewhat modest in our original goals for butane’s policy abstraction and we have you know things we know we could do that are uh fancier and so I think to take that next step um it may be that uh understanding you know that that the system as a whole is is working well could be done in terms of routing algebbras cool all right well maybe that’s a good place to stop thank you so much thank you it’s been a lot of fun you’ll find a complete transcript of the episode along with show notes and links at signalsandthreads.com.