The Science Of Learning Math And Anything Else With Justin Skycak

For those that know me, you know I’m a big learning fan. Like I love learning to a ridiculous degree. I read that one book called the brain that changed itself way back before I started college. And it was one of the moment that I decided that I really wanted to understand learning. I wanted to learn about learning. It got me into a whole journey of trying to understand how the brain was working and ultimately into neuroscience and AI. The one thing that I realized throughout this journey is that there are better ways to learn than others. That there are some fundamental rules that constrain our learning dictated by how our brain is structured. Understanding these rules to learn topic like math, computer science and physics greatly help in speeding up learning process. Now I got the chance to interview the learning legend Justin Skyac on a whole lot of topics around learning technical stuff like math efficiently. What I like about Justin learning journey is that he doesn’t come from like an academic background and family. His way of learning at least at first was very self-directed and throughout his studies he discovered great ways to learn that are actually grounded in the literature. He applied all of these learning techniques uh throughout his studies while he was teaching and mentoring as students while he was doing research and right now while he’s building AIdriven system for Matt Academy where is currently the chief quant director of analytic big title I’ve put a whole lot of chapters throughout the interview so feel free to move around uh the different segment of the discussion it’s around it’s around 4 hours so please take a day off from school or work call in sick and just brew about like 2 liter of coffee or tea while you watch this uh this thing. We’re going to cover is learning backstory, what cognitive science says about the fastest path to mastery, why you think learning math is a lot like training for a sport, and his practical advice for adults, kids, old people, people with ADHD, researcher, we got it covered for everyone in this interview. Enjoy folks and big thanks to everyone that contributed uh the question. It really helps. Justin, first of all, thank you for uh being here. Uh really a pleasure. H can you give us like a high level introduction about like uh yourself uh and then what you’re working at the moment?

Yeah, sure. So, uh my name is Justin Skyac. I’m the chief quant and director of analytics at Math Academy. We’re an adaptive uh online math learning platform, elementary school through university. Our goal is to cover all of math and math adjacent stuff. uh and we take a a full nerd approach to learning. So, so um just trying to optimize at every moment uh make sure the student is working on the the learning tasks that are the the most efficient use of their time. We’re all about learning efficiency. And so what uh what I’m kind of working on right now, what we’re all working on and has consumed my life for the past uh several months, if not years, is is um taking us from a a a workshop to a factory turn transition where we can really turn out a lot of amazing content at the at the same level of quality that that we have been turning out for for years. Trying to floor the gas on this thing. So, um, I used to, you know, in the podcast we’ll get into a lot of stories about like, uh, our our own probably your learning background, my learning background, how how we found, uh, kind of where we’re at right now. Uh, but nowadays, it’s like, uh, it’s really uh, startup early stage company grind mode. And that’s just like my life has just been totally engulfed by by this tornado in a good way. It brings together a lot of stuff. But, uh, nevertheless, like, uh, there’s not a whole lot of, uh, free time for for for for learning, just just for fun anymore. It’s like, let’s go, let’s go, let’s go. And like, uh, um, it’s I find it’s the most uh, rewarding that sort of pressure when it’s for something that actually is working like when you’re trying it out and it’s nothing is giving you feedback, it’s a bit harder. But once you like you have this feedback from the environment that like yes it’s actually is useful and people are loving it and like there’s there’s growth and all that um it can be high pressure but still super rewarding. Totally. Well, there’s that saying, right? Nothing succeeds like success, which I mean on the surface sounds kind of dumb, kind of like well done like but really it’s like uh whether it’s uh building a company, working on a project, learning math, learning anything, it’s like the more the as you experience these small wins, you get a lot of motivation to continue doing it. and and and that that motivation. It’s it’s at least for me that’s been one of the the best sources of motivation is just like knowing that hey this is working and it can work even better if I keep doing it and do it more efficiently and just see what exactly and you see the path clearly also like uh like the the the the environment that you’re in is rewarding you in some area. It’s telling you where you suck like pretty uh directly. So like you know what to improve and like why you’re doing this stuff and why you’re learning this thing in order to do that other element. So talking about like learning and efficiency like I saw in your blog that like you self-studied like 3,000 hours of college math in high school, right? Um, which is absolutely kind of just just like even if we remove the mat just like this amount of hours. I think like even the average high schooler might not go through through that amount. Like first of all like I want to start off but but like where did that passion to like actually do that stuff come from? Like uh like I saw that your background like you’re you’re you don’t have like a like peers around you that were academic or some sort of researcher or stuff like that. was the main driver here. Right. So, this was kind of like a chance event. Um, and it was kind of like, you know, there’s I’d always liked math class uh growing up in middle school and like I I it was it was my favorite class. Math was my favorite class, but am I going to go learn math outside of class? Like that thought just never really occurred to me. I don’t I was not that that much in into math. But at at some point um in high school actually um the summer after 10th grade um I I had just finished up pre-calculus and it was a summer and I was actually getting kind of excited about learning calculus because calculus is like you know when you watch like a sci-fi movie and like there’s always a scientist on the board there’s got to be the integral signs the derivatives like that’s kind of I mean just just at a very surface level it’s like what’s advanced math for somebody who’s going in through high school or like learning pre-calc or algebra like it’s calculus like that’s the first thing at least for me that came to mind and so knowing that I was just you know hey I’m going to learn this next year like I I just finished pre-calculus calculus was next up we we did a little bit of uh calculus at the end of pre-calc and I was just like I was I was getting ready to go I was really interested in like science like I I knew I was going to go into STEM in some form um and it was just like I was getting a little antsy so I’m like why why don’t Hey, like I don’t know how this is going to go, but why don’t I just try and teach it myself? Like what’s what’s the way? Like I mean I guess I’ll learn it next year at school, but like if I could just teach it my teach it to myself over the summer that that’d be awesome. So um so it was just this idea. It wasn’t it and nobody really planted that in my head. It was just one of those things where I just thinking one day I was like why don’t I just try and do this? Um but yeah um that ended up turning into this this kind of chance event spark that kind of led to the trajectory that I’m in now and my my my obsession with efficient learning and and and and so how it kind of ended up is is I found this pretty well sequenced calculus course. It was it was almost like a a proto math academy type thing. It was in some like online course uh collection um like the national repository of online courses and it was like this you you’d go in and there’d be like a couple it was broken up into like maybe 50 or 60 or or more uh lessons the the AP calculus course you’d go in you’d read uh a little bit uh on some slide of information and then you you’d uh solve some problems and it very concrete problems. Uh you got a lot of reps and stuff. So, it was it was a it was a great course. Unfortunately, it it no longer exists. It was like sold to somebody and revamped and whatever. And it’s just I mean I I had to Yeah. Yeah. So, you can’t just like uh go learn it from there anymore. But, um but this ended up working uh really really well. Actually, way better than I thought it was going to go. I thought I was going to spend like the whole summer just grinding calculus and and maybe you know maybe 1 hour a day and and and come out and and have learned like the first third or something. Um but actually like just like we were talking about uh success uh breeds success like having this this small win stack up like got through lesson one, got through lesson two, solving problems. Hey, this is not a big deal. This is actually I like it. This is fun. And and so it was it was actually so fun that I was, you know, I I I used to play more more video games and so math took the place of video games and like it was just um it almost felt like like uh progressing through levels and in in in a in a video game. And uh and so I just kind of leaned into it and I just holed up in my room working on calculus problems for like 8 hours a day or something cuz I was just obsessed. And so my parents are just like my mom’s an art major uh in school. My dad is a business major uh just logistics for healthcare company. Like they don’t and and nobody in my family really was technical or mathematical or anything. So I’m just like it’s it’s like they’re talking with other parents and other like you know like what’s your kid doing over the summer? Like oh he’s just like in his room like learning calculus. So it’s like almost like as if I were just like in the basement playing a video game and uh and and but like nobody says like like oh yeah my kids in the basement playing a video game like eight hours a day over the summer. It’s great. Like did they did they try to tell you like to go out and like do other stuff or like what what were they just really confused honestly. It was uh because it’s like I mean it started out like like wow my my my kid is learning like taking in his education into his own hands and he’s like uh prepping for the upcoming school year. That’s great. And then just like like wait where is he? He’s still he’s still in his room. He’s just like working out these problems all day. like it’s a kind of weird and I mean they they had they had never learned this kind of stuff uh themselves and and they and nobody had like STEM careers or anything but so it was one of those things that was like well it seems like it’s a good thing but it also seems like he’s like addicted and obsessed to it but it also seems like a good thing so what do you what do you do? So they they don’t really like intervene uh or anything which I think ended up being a being a good thing. I was very very unbalanced that summer, but it ended up like can you can you walk us through like okay like I guess 3,000 hours like throughout like this whole like journey like what did it look like dayto day like what did you look like dayto day? So the 3,000 hour so after so that that summer I was spent like maybe 30 days like eight hours a day just like grinding this calculus course and I was just by the end of this I was just so sucked in like I want math. Give me more math. What’s next? like I can’t it’s it’s like um yeah it just felt like I I had I had found this this this this hack or something like no nobody told me like like you can learn outside of class and I’m like okay let’s do this is working out amazingly well I’m having fun this is going to be great for the future I don’t really know exactly what it’s going to do but it seems like it’s going to do good things and so I was like okay so let’s keep this going let’s learn what’s what’s after calculus uh I finished one month uh in in into the summer and I’m like Okay, I don’t know what’s after calculus. I need to look up online like what course comes after calculus? What’s math above that? Is there math above that? I’m sure there’s math above that, but I don’t know what it is. So, I stumbled into MIT open courseware. Um because that has a nice like, you know, a sequencing of courses. It’s it’s like, well, you just what comes after calculus? Well, just look at the MIT undergrad math major that kind of tells you what to do. And so that’s that’s um when I started learning like linear algebra and multivariable calculus and uh and I would just work through the I mean it was it was much less structured than this this original uh AP calculus course that I had found um cuz MIT open courseware at the time that was really just a collection of lecture videos and and some problem sets. Now I know they’ve built out some of their foundational courses a little more uh since then uh augmented it with more learning materials but for that time it was very very rough writing like here’s like 20 lectures and here’s all the the problem sets go and so I just did that and I I found um the some PDFs of the of the textbooks and uh yeah was just doing that. So, I uh did that the rest of the summer into the school year. When I got to school, uh the that following uh like 11th grade, I was just, you know, um this whole self-study experience was making me realize how much time I was wasting in school just kind of sitting there in class. And uh and and so I was like, well, I’m not going to go back to that. I’m going to keep leaning into this thing. So, I would take the the math books with me to like all my classes and just be doing that instead while trying to like look like I’m paying attention and and stuff not getting confiscated and stuff like that. But, uh yeah, so I kept that going the following year and um yeah, you know, it was it was this 3,000 hours uh thing. Um it was way more time than I really needed if I were working efficiently on learning this stuff. But I was I mean as as most teenagers like I think I know everything. I know how to learn like I I did not do my research on how to learn efficiently. I’m I’m just like okay I’m going to go do it. And so I I ended up hitting my head on basically every sort of ledge there was to hit my head on along the way. like um like I kind of just discovering a lot of these um principles like in hindsight really obvious principles of learning science but like you know there was I remember there was like a a day or so that I was that I initially came into it thinking like you know like why am I working all these problems? I can just watch all the videos and learn it that way. And then so I just like took a day and I just like blew through like like what I thought was like a third of the of the course like watching videos and I’m like yeah this is such a hack like I can I don’t have to do the problems I can just do the videos and then I started like the following day just try to like work out some problems and realize like I didn’t I didn’t retain anything from it. And so that was just the learning of like okay I have to do problems and and so every basically every single cognitive science principle um I I violated initially during my self-study and and kind of learned from that and but I I mean um like you said you could compress it to like 70 750 hours something like that with a good design system but don’t you think like the this other chunk actually was about the metalarning uh game and then this chunk the 700 hours was about the math skill because like like uh I don’t know because I went to the same thing right I was doing like uh I was a really into learning at the at the intense level and um I felt like this was the kind of uh metalarning that you have to like see for yourself a bit right a bit like um I don’t know um in sport or art like people could tell you the stuff right and tell you like you should do it this way and stuff like that but then you will not understand what they’re saying until you actually experience it and then you’re like yeah actually yeah I need to sleep right or like I should stay dated otherwise like nothing works right that sort of stuff right like this thing around the learning um which is the core right and after that you can apply this kind of um skill that you learn in in other stuff, right? Because like the the principle are kind of the same all across and you’re trying to map the principle to like the the thing that you have and then the thing that you want to learn. Um do do you think there’s any of this or like you could have learned also the the mental learning skill uh a bit faster, you know? So I mean on one hand I I I I think there was a a little bit of value to having tried some inefficient forms of of of learning and only to realize like hey this doesn’t work. And I think um in any discipline that you are like seriously going into that’s going to be like your main this thing that you’re doing. It’s it’s worth exploring the space of uh you know what just try things out what works what doesn’t. But I just think about the spending 3,000 hours to um you know get from from calculus all the way up through I think I I I I got through about like real analysis, abstract algebra, you know, junior level um math major courses and also some physics and stuff. if I had um you know there was just a lot of time that that I had wasted just kind of being confused doing inefficient techniques that did not contribute to this this holistic view of learning a a little bit a a little taste of like you know did it wrong okay correct to this um there’s maybe some small amount maybe like a couple percent of the time I I would say was like um kind of useful in exploring the space But there came to to a point where I was really like I just want to go fast and learn this stuff. And it was like trudging through mud. And the higher the higher I got um the more things would come back to bite me. Like not doing a proper review system um and trying just working on problems that were like way too hard. A lot of this stuff I I I mean some stuff I figured out on the fly, but a lot of it I I really did not learn until years later how to do this really really efficiently. And if there’s one thing that I wish that I had more during um my my my my teenage years and uh just like as a student would have been guidance on just how just tell me just tell me how to do that. like I just want to learn math and I’m fighting my way through hitting my head on all these ledges. Maybe some amount a small amount of that is is is um kind of shaping my view of like understanding what what works and what doesn’t, but most the vast majority of that time um for me at least felt like it was just it had just gone to waste. And um yeah, there there’s some instances of uh like I also um like my part of my goal in this was like I I want to like just learn the core body of math so I can contribute to like do some cool research projects. And um there came a point like around, you know, uh maybe like 2,000 hours in or so I’m just like, you know, I think I’ve learned enough uh and I’m going to do research. And what was this based on? This was just based on the the sheer volume of time that I had put into learning, not based on how far I had gone. And so I ended up doing a research project which was um just kind of like on my own. Uh it was it had to do with finding a generalized formula for partial fractions decompositions and I spent like a whole summer on it. And I mean it was it was kind of fun. Um, but what ended up happening is I was kind of groping around in the dark in a direction that I would later find out was just made completely trivial by some results in complex analysis. And complex analysis, I mean, it’s it’s a highle undergrad thing, but it’s not exactly super advanced math. And so, it’s like, well, if I just you’re trying to rediscover stuff that were already known. Um, exactly. if you just had like went and then looked at it and you’re like, “Yeah, that’s the shape of it and then you use it like that and okay, now I understand the the setup.” Okay, I get the I get your your your thought process here a lot because um it seems that you were like um almost like discovering the whole cognitive science field on your own and then like trying to figure out how to learn uh on your own. and you like mostly rediscovered like a concept that were already proven. Um so like did you got into cognitive science at that time or like afterward like uh what happened here? Um so cognitive science uh actually was not really on my radar until well I guess a a after high school sort of at the beginning of college I got sort of interested in in neuroscience computational neuroscience and that led to kind of interest in AI and stuff and that I mean it was kind of like cognitive science adjacent um and um but it was really what what really kind of flipped the switch for me was was once I was responsible for other people’s learning. So, starting out um you know, I I did a lot of tutoring in high school also and and got uh I really liked uh tutoring and just math education in general. So, at at at one point I uh actually during college, I I had a a full-time data science job. And after graduating college, I was like, you know what, screw it. We’re not doing data science anymore. we’re doing math education and we’re going to like find some way to incorporate all all of the coding and whatever that I had enjoyed into math education. But what it was really once I once I had a classroom of students who are kind of depending on me to set up some kind of structure um that is going to result in them learning this material. That’s when I I realized a lot of things uh that were some of the less obvious stuff um like the the importance of um you know space spaced review and uh interle and and and and stuff. Um and so that that’s about the time that I that I also met um Jason Roberts, founder of Math Academy. He and his wife Sandy founded it together. Um and and so I was teaching in in the the original school program there and and so he he had done a lot of uh you know just reading around the subject of cognitive science. It was incorporating those principles and so I was like oh wow this is really interesting. Um and for for once like it’s like okay this is the guidance that I was like that I didn’t have. I I was like groping around discovered maybe like a third of it on my own. It was like 2/3 at least that was just like, “Oh my goodness, that makes total sense.” I like, “Okay, let do that, do that, do that.” Um, uh, and and and it was working in the classroom, too. You know, the more that we would leverage these cognitive science principles, the the more that the better the students would would learn the the material. Yeah. So, um and then at at uh some point after after that actually around like maybe like uh 3 years ago or so, um I Jason was like, “Hey, we need to write up about like how the science of math academy works. Can you just make a little like some information for the web page and I I started on that and uh looked up like about 400 pages later, we had the Math Academy way, this big book on how we we apply the cognitive science learning principles. This is cool. And that was I think like the when you’re teaching the metaarning skills become more obvious to you because like that that’s the thing you’re doing. You’re you’re like trying to make this other thing learn, right? Um, so like it’s it’s not really about the the skill anymore like the the math biology or like this is just something that goes into the scaffold that you’re going to go and then uh like uh make the student learn um because like there’s two things like in my view right however skilled you are at the thing that you’re teaching if you don’t know how like learning is happening um you will have a hard time teaching like I’ve seen like a research level uh uh professor like being completely useless in the classroom like actually actually useless they will they will we’re doing more harm than good I know that because I was tutoring this class and I was like what the what is happening right um and then the second thing is that like the learning is not coming from the teacher to the student it it is emerging from the student so you have to make this emerge right so like it’s not like you’re going to hammer them up with like whatever and then you just bulldoze them if they don’t want to learn they won’t they won’t learn right they absolutely will not learn so there’s these two component that comes in I think like the whole like field of coive science and like psychology neuroscience um they add plenty of time to figure out like how to make this thing more uh efficient um so it’s kind of a it’s kind of worthless to to like spend time rediscovering stuff that is already uh working um I wanted to ask you. Uh, sorry, Greg. Oh, yeah. I was just going to say, um, yeah, I totally agree with, uh, I’ve had that same experience with, you know, professor, highle professors. Like, you you kind of, right, you sign up for the class, you’re like, “Wow, this professor is like really good at research. Like, I’m going to learn so much.” And then you you get in there and and then you you get served up these like these slam dunk problems that are like supposed to the professor thinks like oh I’m going to create these like really just a a few really nice like really interesting problems and give them to the students and you get those problems you’re like what is this like I don’t even know where to begin and you just are flailing around and and the professor is you know not really helping you bridge the gap from from from you to them and I’ I’ve seen seen this too in a lot of students I tutored as well. Yeah, it’s it’s and I’ve seen it a lot in um in engineering like actually like it was plague in like I was ting a lot of engineers and um sorry student engineers and like for some reason there there was this this kind of elitism in engineering where it was okay if the class some part of the class was not understanding anything. Um and like they they had this kind of uh weeding out of the people and stuff mentality. U which in my view was kind of absolutely nonsensical like uh it is possible that all the class can get 100% in your thing if you like if you’re able to teach it properly. Like it is possible. It should be something that is in your mind and that you should be able to kind of like put yourself back into a question whether like what you’re teaching is actually good or not, right? Um but uh yeah, we just kind of accepted that uh right the the class moves in lock step and and there’s a normal distribution of grades at at the end and some of these grades are failing. Some of them are like just barely understands the material. A few grades are like okay they they they get it. we just kind of accepted this the system but really yeah if you if you have um you know you you have every student working on what is the most efficient use of their time um that’s the thing like if you look uh in this uh sort of system the one that are outlier and they like the the teacher is like look at these like it could be any teacher they would have done this because they’re studying on their own and they figured that out right and I know that because most of the time it was me like I was like going into these classes and I was like this teacher doesn’t know what he’s doing so I cannot just listen. I’m wasting time. So I was like doing the textbook. I was like getting my hands on like any past exam that was legally possible any like hint of a problem I could solve and I was drilling them and then drilling the teacher to show me like what he would have done in this case if it was like a something that you have to to work through. And then I was I was doing okay because I didn’t I was was not just listening to these guys like uh because there was nothing of value here. I was listening to the material, learning the material and then trying to apply it to how they were grading it. Um which is totally different than what everybody else was doing. They were waiting for the material to kind of like be taught but it was not taught any shape or form. Okay. Hey, wait. I had a question about um your research because you did some uh research as a teenager. Um and I wanted to to understand like um like the fundamental math stuff you were doing, right? And the base that you set up like did it help like uh like push to those research project like uh especially at the Fermy lab and the and the certain project you were doing? Yeah. So I I’ll talk a bit about that. So the Furby lab and certain projects these were projects that I did with local universities. uh formula lab was with um Indiana University Southbend um where I grew up uh and uh in in the professor Elon Lavine’s lab and then the CERN project was with uh a lab at Notre Dame uh that’s quarknet program which connected um like some of the top high school students in the areas with working on um projects that were were going on with uh some labs there at Notre Dame and and so so the math It it was it’s kind of funny. It it it it helped in a lot of big ways um get my foot in the door uh in terms of these projects um because you know it’s you learn a bunch of advanced math you’re instantly you stand out um as in in my high school as like okay this who who who are we going to pick they could pick one student to to send to this this quarknet program at Notre Dame and they’re like well who are we going to pick? Well, Justin is like just crushing the like calculus course as a like a grade early or whatever. Like I I guess and he’s like seems to be like learning physics on his own. Like I don’t know, send him. So that was Yeah, it was just an obvious dec because that’s what happens when you get like really like you know you find your your obsession, you get really ahead, people take notice and you get a lot of opportunities to pursue that further. True. Um and um with the other uh project, so so my my junior year I was I was um working on the the Firmula project and that was actually just uh I I I I was taking this this uh research class in in my high school where the the the point was to reach out to local professors and uh and just try to get your foot in the door working on a research project with them. And and so um part of the thing that that you know made me seem like hey this kid is is gonna actually be able to help contribute a little and not just be a total waste of time was the fact that I had like I already had my math foundations in place for all this stuff and I had taken had found myself to to learn. And so when I when I got to these um you know started working uh with these labs, it was you know math was just a nonissue. I remember for so many for so many other students in uh this this um in my high school research class like they they would go off to their their labs as well during the school day and uh just for like a at the end of the school day for for an hour is the last period of of school. So you’d you’d go out and leave early. Um, so a lot of other people would would be struggling with with some of the math involved sometimes, you know, like they maybe somebody would would get into a physics lab and they would not only have to spin up on the context that the lab is operating in, like what research are we doing, what are the materials, but like there’s just a big mathematical gap that was placing hard limits on this type of stuff that they could do. Um, now I didn’t have that gap and so that was good. Um though I I did have some some gaps in other areas like I mean mo most of my my my projects uh had to do with experimental physics. So there was like a you know math was one side of it but there was also you know the the firm lab project I was making a trying to make a material to to better transmit sound and I had never like I was not a very hands-on lab kind of person. So I that was a that was definitely a point of like oh man this is challenging but in in in terms of uh you know ma math just became a a non-pro and um and I I I think um to anyone who who wants to do uh research especially high school research it’s like removing skill gaps is such a big um advantage there because you can you know just imagine like say there’s some lab it’s working in some setting some research setting. If you have all your foundational skills in place, um then you can just like take off running and be a a serious contributing member of the lab. Every every little pillar um uh of foundational skills that you don’t have in place is going to go is going to slow you down. So So I would say in my case, like okay, the math was a pillar that I just knocked out of the park. That was like a non-issue. some of the more hands-on experimental physics stuff uh was was definitely more of a more of a drag factor for me because I I’d never um trained up, you know, that that kind of hands-on. But this is super interesting. Uh and like did you improve in like uh this exper experimental physic like a manipulation type of stuff you had to do during that? Did you try to like learn it and and train with it like how did it look like that part, you know? So I I could have leaned into this a lot more. So that I I came in wanting to do theoretical physics, right? So I was like, I don’t want to do experimental physics, but I want to do physics. And like I mean, you’re a high schooler. You’re like, hey, can I get involved in this physics stuff? Like then they’re like, yeah, you’ll take whatever on this project. So yeah. They’re like what? And I’m like like I’m I’m not about to like turn down experimental physics opportunity to say like no, I want to do theoretical like that or nothing. And they’re just like get out of here, kid. What are you talking about? like so. Um, so I was a little like, you know, doing these these I had not quite found my my really good fit with these physics projects. It was like it was like a halfway good fit, right? Cuz I like that math pillar like, okay, that was good. But the experimental physics physics pillar, I was just, you know, I I didn’t I wasn’t really interested in that kind of stuff. And um yeah, I I did not um take it upon myself to to learn this stuff out outside of lab. And uh so it’s it was kind of like I I just did kind of, you know, run-of-the-mill projects there. And this was enough to to I I ended up making it a junior year to the Intel international science and engineering fair. And and so the this the project was it was like it it was good enough to to get to that level but it was not like good enough to like you know get place uh like get get a medal or result in a like a paper or you know stuff like that or or even like kind of really set a direction for me to like I want to continue like looking into this more. to be honest like um there’s a whole lot of luck involved in like when you are a junior and then you enter in a lab and then like you get a paper out of it it’s kind of you you got in there at the right time I know that because my first paper was that like I I got into a molecular biology lab no experience at all zero right I was I was not even like in in molecular uh or like cell stuff yet I was more in the math physics When I came into the lab, I just wanted to to work on like uh learning memory and on the molecular level. And I found a lab that was super interesting. Read their paper and I was like, can I just help? And then they were like, “Yeah.” So, I got in and to be honest, I think I wasted them so much dollar by just [ __ ] up their recipe again and again. Um, it was crazy. There was like so many steps. I had to splice genes like to change them. Then I had to go make them sequence and stuff and then I had to like grow the bacteria and then like all this I was messing up like on the daily, right? Uh then I was like I can’t just I can’t just mess up their old old thing and then like drop out of this. So there was this like lab assistant that was there. She was super old and I was just like asking her all the questions. Um and um it turned out at the end like I got some good result. Uh and I got into a big paper. My thing was in like their gigantic paper that they already was 95% done. So by chance I got in there and my first like publication was journal of biochemistry or something like that. Um I to be 100% honest I didn’t understand like I would I would say like u u 30% of what they were doing like u I understand my part and like how it kind of fitted but the whole story was like this is that bit beyond me. That’s interesting. I actually have a question for you on on that. So you you joined this this uh molecular biology lab and you you did did you mention you you had strong like math uh and physics foundations by that time. So when you were uh joining the lab was um you know was the math and phys were you leveraging the the math and physics in your stuff? Absolutely not or Okay. It was just like very pure biology like methodology. Okay. That’s interesting. Yeah. To be honest I could I could have been like a art major or like just a magician and like it would have made no difference. I had no clue what like when I got in I had like I had some concept about the cells and stuff. I’m like yeah but um I was so many layer of abstraction deep into this thing um like I was literally like I was running my my experiment make sure I didn’t mess it up and then I had like 3 hours of incubation or like waiting for the autoclav to finish cleaning my [ __ ] So I had to uh I was like blitzing through all their paper, the related paper, like pestering absolutely everybody including the other lab that was next to to us about the dumbest question you could ever imagine, right? Like literally they were listening to my question said like you didn’t know that and I was like I dude I don’t know why they got me they let me into this lab, right? Um but but by the end of it what I was able to kind of foot find a foothold, right? And then then I I understood like a a whole lot of thing right and I also understood that like in molecular biology like specifically it’s a bunch of recipe right and like the recipe is a certain way and if you look at the different lab they do the stuff a bit differently right and all has to do with like the history of the stuff and some mistakes with like living things are just like pure chance like your thing is messed up right and like one I remember one time my thing was messed up three time in a row but the bacteria culture I was trying to like make grow and not die at some point like it got really weird and I was showing this to the the supervisor always in in his office like I was just walking in like you need to help me out right you got me in you help me out right and uh he was like oh just ask this other guy like he’s the guy he’s a genius about this stuff okay cool and I asked the dude like long dreads like super chill dude and he was looking at my stuff and he was like yeah this is messed up so I was like why he’s like not sure it could be like a hundred different things and I was like so what do I do like try again and that’s it right so I was like okay so it’s it’s a bit different than like the other more like deterministic thing that I was doing before right like we’re dealing with living things there are some stuff that if you do it like uh uh like wrong for sure you’re going to contaminate your things but there’s other things that like happen because it’s like literally like a more complex living thing that is that is trying to grow right you’re you’re literally you’re shoving a bunch of stuff in bacteria and you’re trying to make them express a gene and then like create protein, right? Like the bacteria is something living like so like sometimes like you don’t want in this specific configuration that you didn’t account for. Yeah. So I learned more variables. I learned a whole bunch on the fly. Um but that was fun. Well, that’s that’s uh you know, just you going and and like pestering these people with with all your your questions, which I guess initially were a little bit uh novice questions, but then you built up this this form of mastery that really takes uh you know, putting ego aside and like, you know, I’m going to ask the dumb questions because I have to learn this stuff cuz you you don’t get results. What’s the choice that I have? Like the choice is I’m going to be dumb, right? Literally like I could instead look dumb, right? And then just tomorrow going to be a bit less, right? And then I’m just going to like climb up of the mud pit that I put myself in. Um, and the the interesting thing is that by about the five months mark because I was during the whole summer. Um, I was actually pretty good at this. Like uh the uh the assistant was like, “Oh, wow. You’re running your experiments so well and like they’re so well sequenced.” I was like, “I’m just doing whatever the heck you’re doing, so thank you.” And then like the supervisor um tacked on to like a a PhD candidate um to take over my work after um uh I was I was out and she was taking note and asking me question. I was like wow I can actually like uh like a like provide something now because I I’ve done everything that is was messed up for like uh six months at that point. It’s very real. Yeah. You know once yeah he has something of value there when taking notes on what and what you’ve learned. Yeah. So, um, this is good, man. I I think like we can maybe switch gear to the stuff that you’re working on right now, which is Meta Academy. And, uh, like I’ve been looking at the stuff. I never actually used it, right? And I’m planning on using it. Um, and I saw that you said that like the you can teach math four times faster. It’s basically like the the system that you were uh using before, but in a more kind of applied and uh, well thought of way. um for someone let’s say like me that never used it, right? And let’s say I don’t know anything about it. Um like what does it look like? What the does the learning section look like? Like how the the system works? Because it’s not like a a classical you watch a video and you do the stuff like what’s what’s the flow looking like here? Yeah. So starts out with the diagnostic test. Figure out what you know, what you don’t know in whatever course you’re trying to take. It’s going to look back for missing prerequisites many years back into math. So, if you’re placing into calculus, it’s going to it’s going to test you on a whole bunch of algebra. Like, you don’t know completing the square. Well, guess what? You’re going to learn completing the square. I’m going to add that to your your learning plan. Makes this custom course for you. Now, you got a custom course that you’re going to go through. This is kind of like your your knowledge graph. Uh and and so we we know what you know, what you’re ready to learn, and what you don’t know. And we’re going to serve you stuff that you are ready to learn. And so there is g this is going to be on your your knowledge frontier we call it your your kind of your your edge of mastery and we’re going to build up your your understanding of of of these topics. Now what does a single learning experience on a topic look like? Well this is a lesson and it’s kind of like a a sequence of minimum effective doses of instruction and you actually solving problems. So, you think like um you know this whole situation is very similar to um if you were to go with a personal trainer, what are they going to do? Well, they’re gonna they’re gonna first just see like, okay, what what are you trying to do? What sport are you prepping for? Or what are you trying to increase your vertical jump or what are you trying to do? Where are your weak points? And what can we do to to to to get you, you know, better at those? They’re going to have exercises for you. and each each exercise they’re going to like you know I’ll demonstrate it once like we’ll talk like this is what it is here’s how you do it okay now you practice and so that’s what a lesson is like it’s a starts out with a little just a brief introduction and uh then we initially uh immediately launch into into problem solving you you see uh okay you got your introduction you this is the type of problem that we’re going to focus on solving here’s an example of like how you how you solve this type of problem okay now you do uh a few problems. If you’re just knocking it out of the park, like you you you answer the first two questions in a row, it’s like, hey, you got it. Let’s move on to the next the next uh more challenging version. So, um if if if you’re if you’re struggling a little bit, you like, oh, first question wrong, next one right, like we’re going to make you solve a few more problems just to just, you know, you got to end on two successes in a row before advancing. Make make me think of a of a tutor actually like uh because that’s what I was doing. I was asking them like like hi nice to meet you like what do you know? And then they were blabbering about like everything and I was like cool. Let’s like do some stuff like and I was looking at like yeah he doesn’t know this stuff. Let’s just like go back and then do this thing. Oh, now you get it. Okay, perfect. Like we’re not going to waste time because you’re literally paying me right now, right? So, we’re going to go and move to the actual stuff that you need to to uh Was this inspired by um by that sort of a of a So, this is this is born out of our hands-on experience teaching, right? And just like uh you know, the the tutor the tutor is really I’m I’m glad you brought that up because the the tutor is the the that’s like our our model organism basically. We’re we’re trying to emulate the the decisions of an expert tutor who knows everything about your um your knowledge profile, like what you know, what you don’t know, and has unlimited computational bandwidth to like have you, you know, it can just make up problems like it’s it’s got a whole inventory of problems that it can match you with and and every single answer that you do, it’s going to kind of take that into consideration to to get you working on the just What is the most efficient use of your time right at this second? Okay, we’re going to we’re going to have you do that. Okay. Okay. Okay. Cool. And you you you uh mentioned like expert knowledge about the stuff like behind this whole thing there’s like a knowledge graph that is mapping like I think two 2,000 some topic. But the interesting bit that got my attention here is that like they were encoded manually like you you like I think you spent like what 200 something hours on like talk about that stuff the whole graph. So we started we started this stuff I mean this prelims right this is like you know there’s just no way to generate any of this information. So, we started this stuff back in uh like 2016, I think is is when um you know, Jason and our uh director of curriculum, Alex Smith, uh started working on this thing. I I I joined around like in in in in 2018. And so, we a lot of this stuff we’re just um you know, we’re handcrafting it manually. Um and to this day everything is is very you know it comes from a human expert just paying very close attention to based on their experiences teaching um and and what they know about um you know uh max efficiency learning and how to get students through material um in in in a very uh in a way that’s guaranteed to work. Uh how do you how do you do it? And so we um we build like a lot of tooling over over the years to make this process more and more efficient. Um so we’re leveraging like all all the tooling that we can. But yeah, ultimately it comes from from a a human domain expert is is is making sure that uh like all of this is is to their standards and um and and and this is all mapped out um by by them. So this is this is uh Alex Alex Smith our director of curriculum. He builds kind of the the forwards knowledge graph of like okay what are the what are the topics that we we want to have students learn? What is this course? This calculus course what what is it? What are all the topics in it? We got about 300 something topics in our calculus course. Each one is like an atomic unit of knowledge consisting of roughly like three or four knowledge points where it’s like it starts out with the simplest version of the problem. um and and then um layers on additional complexity as you you evidence your your ability to solve problems in each successive knowledge point. And so he he maps all this out. He connects everything up. Uh the prerequisites, what what do you need to what what what topics do you need to have learned before we serve you this new thing? And um and I kind of encode this this backwards graph. Uh we call the encompassing graph. and it and it says what skills encompass what other skills, what subsklls are you exercising as components of more advanced skills? And this helps with our our space repetition algorithm, which um is always, you know, when you do a a review on a on a on an advanced skill, you’re kind of implicitly practicing a lot of subs skills. Yeah. So, we’re always trying to, you know, something that we we we we do is we’re always trying to maximize your learning efficiency. So if we can get you practicing your subs skills that are you kind of due for some review, if we can get you practicing those by learning something new, learn a new lesson that knocks those out, then then then we um then the algorithm serves. So it’s like a like a directed a cyclic graph, right? Because there’s no there cannot be any cycle that would not make sense. Um, and then like if you’re working on like this skill that like has the dependency and you like you’re super good at this, this whole kind of like a parent lineage is kind of solved so you don’t need to go back to that. Um, okay cool that this is Yeah, exactly. Um, yeah. And so something about just the whole, you know, approaching this as a so it’s like why not just, you know, generate this just try to generate all all the content with with like an LLM or something. Well, um, so the thing is a lot of this this this content the reason why it works so well is because um it’s based on all of our learnings throughout the the years of doing this manually. And while while LLMs are are like they’re very I mean they are really good at uh traversing this whatever um knowledge that they’ve been trained on, right? everything that’s been written down um on the internet and and uh often like uh applying sequencing of that of that knowledge to to kind of go expand beyond the initial knowledge base. There’s a lot of stuff that you can learn by, you know, just working with students hands-on. Um, and just kind of almost like extracting data from from reality itself, not not just a compressed version of like this is reality according to all the text that’s been written on the internet. There’s a lot of stuff that’s kind of like off the the training that and like this is not mapped anywhere like like like seeing like somebody like a actual human being trying like trying to get like toward whatever they they’re doing in this in a class and seeing like the 100 different way they can struggle uh like firsthand. This is way different than like even like the research about like teaching or like uh the research about learning. It’s way different when you’re looking at this kid right here and how they’re struggling and why they’re struggling. You can decompose this for sure, right? And there’s like emotional component like resistance to this like uh uh they don’t like the material also maybe, right? All of this stuff can can go into into there. Um it’s really something very like very complex, right? And as you see and you see more of these students, you can kind of start to map it out. But like the the I’ve I’ve never seen anywhere like something written like about like the specific like uh uh reaction that they can have to the material. Well, actually just on on what you were talking about like students reaction to material. I mean that’s one of the things that um you know that the the design of minimum effective doses um of instruction and and practice. I mean part of this is just inspired by you know cognitive science of like you you know you active practice. let’s maximize the amount of time that you’re actively solving problems with guidance. But another part is also just inspired by Jason Roberts um founder. He he always talks about like how it’s not you know when he was sitting in math class um it’s not that he hated math it’s that he hated being talked at for an hour. And so just so many we we’ve realized that so many students um especially students like with with um you know ADHD or just who are just like really uh can’t sit still type of type of students like they don’t want to watch a video for an hour. They don’t want to watch an hourong lecture video and if they do it’s often because they are just want to like stare off into space and not think about it. It’s like what what really keeps the students in engaged with the process is, you know, actively solving problems that are at the right level of difficulty, kind of entering sort of that that that that flow state right at the edge of their ability. Um but I I want to say in addition to like you know um just designing like the lessons in internally to to try to you know keep keep the experience like um you know maybe a little challenging but very achievable for for students. The the other part is like there’s also the the whole connectivity of the knowledge graph. So, this is this is stuff that you probably would not think about if you’re if you’re um if you’re just like trying to to maybe get an LLM to dump out a knowledge graph or just ask somebody else to design a knowledge graph. There’s there’s a lot that comes into play in terms of structuring the knowledge graph to to make students successful. Yeah. And so, um like breaking we think we often think about things like um like quantifying the the the cognitive congestion of various uh nodes in the knowledge graph. Like if you have if you have a topic that has like 10 different prerequisites that are all being pulled in that you’ve never practiced pulling in together before, like that’s going to it’s going to inflate the cognitive load a ton. And so there’s just a a million of these little optimizations that we that we put into to make the the knowledge graphs like really easy to to to learn from. Yeah. And um even when you are teaching right sometimes you don’t have like you don’t remember like there was uh like that much prerequisite and you’re teaching it like hey this should be simple you do this like what’s that right like oh it’s it’s taken from this what’s this thing right and then you’re like okay then we have to go back um the knowledge graph thing is very interesting for me because it is trying to get to like I mean this golden type of learning where you have an expert next to you and you just like guiding you, right, with like an infinite amount of patience, right? And like goodwill, they they will be there like rain or or or shine and they will just like help you get through your stuff, right? Um and I was thinking that like a textbook is also just that, right? A textbook is this, but it’s like handcrafted with one or two others and that’s pretty much this, right? and like they don’t have as much flexibility in how they’re presenting the material because they have to make it like extremely linear. Um, and yeah, you see like I’ve seen this in like the more advanced class, right? Like at some point the chapter kind of doesn’t make any sense anymore because like you like it it does branch out and then there’s like okay this is like also for these three topic and then by the way this one you require these two and then this other and then like the later chapter is always just like yeah good luck kid right like go ahead and like best of luck and you always see depending on the textbook some some biases from the others and then some like gap that are in there or like some stuff that are more interested than than others that say they’re interested in applied stuff. But I I I it seems to me that this knowledge u uh graph is like a deconstruction of this stuff and then you’re of course injecting like the the knowledge of the the expert because they are acting as abso like a source of truth for uh the the knowledge of the oneonone tutor that happens. Yeah, exactly. It’s it’s kind of like um I think that’s a good analogy. It’s like if you took a textbook and let’s say that this textbook was um constructed not just by by one author just doing a brain dump in isolation but but uh an author an expert on the subject who’s also looking at all the other textbooks all the other treatments of the subject trying to just hit all the bases a comprehensive course. Um and so they write this textbook and then instead of presenting it to the student in this linear sequence just here’s the unit on uh I I’ll just use algebra uh as an example. Here’s the unit on linear equations. Okay, now let’s move up to quadratic equations. Okay, now let’s do some trigonometry. Instead of presenting by units, you you really chop it up into the atomic bits of knowledge and and have that way you can you can really trace out the student knowledge as a as a contiguous um as a contiguous area in in the knowledge graph. You know exactly what they’re they’re ready to learn. You can deliver the the next pieces knowledge. you can kind of adapt the number of problems to the the student and and you’re you’re also you you keep it interesting, right? You don’t have to just do one unit for an entire week or two weeks. You know, at at school I often like, you know, this is uh this quarter we’re doing trigonometry, nothing but trigonometry for for like three months. Sick of it, right? Yeah, I know. you get sick of it and it’s like in addition to just getting sick of it, it’s like that’s not even an efficient way to to learn the material. It’s like you ide like ideally you want to be spacing it out, coming back, revisiting it. Um you don’t want to be blocking or like uh having your practice on the consecutive material all the time um in interle all the different units. Yeah. But like a classroom I think is always flawed because like it’s a one to many uh type of teaching, right? It’s not like onetoone like custom made stuff. Um I always find it funny like I brought my wife to one of my class. It was organic chemistry tree or something like that and it’s like it’s like this giant stadium that can hold 700 student and then there’s like this one tiny teacher at like over there. Yeah. Yeah. It’s like an impossible task for one like I mean that’s a good because like 700 students how are you going to you can’t you can’t just one one teacher cannot deliver an optimal learning experience to 700 students simultaneously. It’s very easy to hijack though because like I I was never afraid to ask question because my mentality was like I paid for this stuff I’m going to learn this. So like I was raising my hand and the 700 people um and asking like the dumbest question like uh like that I didn’t get or something like that. Um, and like if you think about it, there’s like 500 people that are like, “Ah, [ __ ] there we go again, right? This stuff that like you didn’t understand this thing, but like it’s very inefficient. Like it’s absolutely uh it’s just a way to scale to like so that the average can go up or something like that, but on an individual level, it’s not that um uh it’s not the best way to um um to learn.” On the topic of knowledge basis, right? Um like there’s the expert curation here. Um do you think that generation capability of LM are useful as an interface uh to the to the knowledge base with a student? I’m not talking about like generating a knowledge graph. Let’s say we have like a very well created knowledge graph made by expert reviewed and like really wellcrafted in a way that like it’s very hard for an AI to kind of like figure this out. Do you think that like having an interface uh has any use of this or like even then it won’t be u won’t be that useful? Well, I think it’s an area that is that is uh it’s on our radar, worth paying attention to. Um at at at this moment there’s there’s a lot of challenges with that sort of setup. It’s it’s one of those things that’s like I mean there’s there’s a pull, right? You want to imagine like you think of efficient learning like of you think of this personalization this glove fit you often imagine like some some tutor having a personalized conversation with the student around their their interests relating the material to their their interest and stuff like that. um now that that can potentially work um really well in some cases, but you know there’s there’s a lot of surface area for things to go off the rails. And so just to just to name a few a few um cases um sometimes students um want to kind of get out of scope of a of a topic, you know, like you’re teaching one thing and they’re like, “Oh, what if you do this? What if you do that?” Blah blah blah blah. And so some amount of to this can be like um can enrich the the learning experience, but there there also comes time when it’s like, you know, kid, like I know you’re excited about this stuff, but we’re just talking in the abstract right now. You know, I I I I used to uh back when I was teaching, I I would have some students sometimes who would ask me like tons of questions about like we’d be learning like introductory calculus stuff and then they’d start asking me questions about like what’s the what’s the hardest integral to solve or just like what’s are all all integrals solvable? Like how do you know or how do you not know? And you know, pretty quickly you get into a realm where you’re no longer really like doing skills-based teaching. you’re just kind of like talking around the subject uh almost as if they were watching like a like a YouTube video like some visualization or like what’s the hardest math problem or or like you know that that kind of stuff and so it it can be there’s some element of like well okay you got to kind of scope it down to know like there’s there’s a balance to be had and and and and you don’t want to allow this the scope of what you’re doing to to to bleed too much. Another um part is is that um I mean so in addition to to make try like making it non-gameable and and stuff like that. There’s a challenge in in in that um generating content on the fly is always I ls are like stochastic um generators, right? And and so you you don’t I mean you can they’ve gotten a lot better at all the like uh hallucinations and stuff. There’s definitely like a or order of magnitude improvement in that if you don’t have static content underlying it’s it’s uh it’s a lot harder to yeah writing is the gamble. So and um and additionally just just to name one other thing um one thing we do a lot with our knowledge graph is that’s an advantage of having um this kind of static content is is analytics on you know every student is being served the the same exact stepbystep lesson that we have been optimizing for years to be the most like assuming the students have mastered the prerequisites leading into that lesson. this is the most efficient most robust way of of of getting a student to to learn the material in that topic. And so a lot of these these optimizations, you know, maybe we had three knowledge points and knowledge point number two was actually a little too aggressive and uh instead of 98% of the students getting through it on the first try, it’s only like 80% or 70%. And then we we break that up into two steps and like just you know we we do all this content has gone through so so much refinement an analytics refinement and um once you kind of get rid of like this this idea of of static content it makes it a lot more difficult to to do analytics. Yeah. Not to say that this could never work just just to say there’s still lots of challenges. That’s uh like um because I uh I built like AI system for like a uh production grade and stuff like that. The one part I am always looking for for when to apply like uh like LM based uh generation is if I can get like this shape of um summarization, right? If you can get like the base ground truth and then for some in some shape or form, right, able to kind of go down like this, it almost always work, right? It almost always work. And like you have very little guard to put because they will summarize, right? And then they’re very happy to just summarize and not do anything else than than do that. And like even though they can they can like I’ll put it in like conversational way like like put on different mask they are still just literally summarizing the content. So in your case, I think like having this like static graph that like no AI is able to touch it, right? And then having like this kind of thin um uh like a layer between the uh static content and the user. Um you can literally leverage one of their best quality which is to be like extremely psychopantic, right? like they they like they will like rope in the the user in the in the conversation and kind of deliver the content in like um how it like a a way that like it’s more amendable to to to this specific person because like I remember when I was tutotoring I could not talk the same way to everybody right there’s some like I had to like do it in a different way right I had some I had like to like take different example right or like bring them to like I said like ask them more questions right than uh than than others others just like they try it and then we had just this feedback loop other literally had to be taken by the hand for the first because they were kind of frightened by the problem and then we do it again with a different variation but it’s the same stuff right and then we do it again and then they feel comfortable and then like you know we can move on and stuff so it’s this uh part that was I was um I was thinking about so food for thought maybe maybe it’s a terrible idea no I mean it’s it’s definitely Definitely something that’s uh I think will remain on our radar uh is is like you know the idea of I mean it it’d be cool to um I mean we’ve we talked about the idea of like yeah you know some people when when when content is is presented in a way like making analogies or using word problems in a setting that they’re just really interested in like uh like oh they’re interested in in soccer like okay let’s make this word problem we’re doing all our word problems about soccer. Let’s let’s just make it fun like like like that like that can definitely um elevate the the learning experience. Um there’s other there’s um other elements just to say not not that you’ve suggested this but a but a a common uh confusion though on on this point is like some people think that students need like a million different explanations of a of a topic until one like clicks and and and so that’s what I’m used to hearing uh in terms of like you know suggestions for for for using LLMs in the content. But what we found is that if you just make sure students have all of their prerequisites in place leading into the lesson that you’re you’re asking them to learn, um then they’re just prepared for it. And and often times like what you experience as a tutor, like when you’re when you have to explain things a million different ways for a student to get it is is really Yeah. there’s something missing and you’re just kind of like trying to cover the space to just hit what is this missing thing? I don’t know. I mean like just like rapid fire like to try to hit it which I mean if you if you’re if you uh don’t have a mastery learning system that is guaranteeing for because it’s something in place like that’s not a bad heristic. Yeah true. Yeah. But um we are just on on the topic of uh you know framing things in in ways that are like really you know exciting to students. One of the things that that we are actually have uh coming out in in one of our upcoming courses is um is projects that are just based in um you know just really interesting scenarios at least that we think are very interesting. So it’s a little you know uh interesting to people who are interested in um astrophysics modeling uh planning out civilization on on Mars uh translating writings from longlost civilizations stuff like that. So we got a lot of um cool projects like that. But um yeah I mean the the the most where this path leads Yeah. is is is is kind of like matching like okay what kind of projects are you really interested in to to the student. That makes that makes sense. Uh well a lot of stuff to to try out here. Um before moving on into the rest of the stuff I wanted to dive into like um uh your space repetitional algorithm that you you injected into this system because like that’s uh like a you’re not just going to learn it once see it and like check mark it’s done. you kind of have to practice it again and see like checking that you’re you’re moving in the right direction. Um so you have this kind of firepace repetition algorithm um which is different at like flashcard. I’ve used flashcard a lot right in my um uh in the different classes I was I was uh I was taking but like in this specific case you’re uh you’re leveraging like the the knowledge graph the fact that it’s likeical in nature. Can you like walk us through like high level like how this specific algorithm work and how it’s different from like just a like anky like a flashc card based system? Yeah. Yeah. Yeah. So imagine that you want to you want to learn math. You know you got to do some form of space repetition because the spacing effect is one of the the few like free lunches you get in in learning. uh if you space out the material and serve it at the right time, you can kind of minimize the amount of review that you have to do while while maintaining a certain level of retention. So how do you do that? Well, option one, well just you know this is this is a thought process that kind of leads to this this this algorithm. you know, maybe you say like, okay, I’m doing all these math problems and I got to make up Anky formulas for them or Anky flash cards. And so, so the first thing I think about is like, well, what what do I put on the flash card? Um, on one hand, like, yeah, there’s some formulas and stuff in math, but so much of math is like if you just memorize the formulas, that doesn’t mean you know how to apply them or how to solve problems with them, right? So it’s like, well, how do you do Anki for actual problem solving? Well, okay, the solution to that is just put problems on the flash cards and just have a system that like instead of serving you the same problem, it just replaces it with a slightly different problem. And so those are kind of your your your repetitions that you’re doing. So now what you run into pretty quickly uh in the setting is um this this thing that review hell basically is like people already get this with with flash cards, right? That just like ones that they can go through in like 2 seconds, 3 seconds is they pile up. You got so many flash cards to do and um even though like you can get through them so quickly now you you’ve just made this mathematical problem solving, right? So now it takes you like a minute and two minutes to go through the go through each flash card. And so now you got this massive backlog of review to do. So what do you do? Well, well, so if you you start thinking about it like, okay, you got this massive say say you’re in this situation. You got this massive backlog, right? And you got to practice uh solving uh a linear equation. You got to practice solving a quadratic equation. um got to practice factor. Wait a second. If I factor if if I if I solve this quadratic by factoring it into linear pieces and then solving uh each equation arising from the linear piece, I’ve effectively practiced solving linear equations. If I solve a linear equation um 2x + 3 = 7, I am practicing subtracting and dividing or yeah, that kind of stuff. So you’re like, “Wait, I don’t have to do all these cards. If I do this card, if I you just take a card at random. Okay, I did this card and so I can remove these other cards from the pile because I’ve effectively reviewed those as subs skills.” So So that now that that starts to get into um you know, compressing your your reviews because the next thing that you want to do after that is like, wait, I don’t want to just draw cards at random from the pile that I have to do. I don’t even want to necessarily draw the card that is most overdue. I want to draw the card that hits the most the biggest number of do repetitions uh with the one problem. What’s the one the golden problem that is going to make the biggest dent in my review pile? And so that’s that’s uh where you kind of get into like the knowledge graph and fire that’s an acronym for fractional implicit repetition. So the the repetitions uh you’re getting implicit repetitions on um on subsklls that you’re practicing uh as a as a component of of each of each more advanced skill. And uh often times these these subs skills are are fractional uh in the sense that like well you solved uh this problem it it didn’t compass encompass 100% of the cases of this subskll like you know maybe uh quadratic equation you factor it you solve the the linear components. Okay, you you’ve you you’ve you’ve handled the like ax + b = c equations or maybe it’s like a a quadratic equation x^2 plus some leading coefficient of one. So you’ve handled the the linear equations that are like x

• 2= 3 like stuff like that. But what about when there’s a coefficient on the x? There’s some cases that you don’t cover. So you kind of track the you know what what skills are how how what what fraction of the subsklls are being uh encompassed. And then you you just as you do each each review problem, these these repetitions uh they they they propagate through the the graph and uh trickle down almost like you can visualize it almost like strikes of lightning. Like you practice this thing, it covers a bunch of stuff below.

It’s as if like you converted this into like a dynamic um programming problem, right? like and you’re leveraging the structure of the graph to kind of like uh reduce the the the effort complexity on the student uh which is cool. This is this is pretty nice. Yeah. Yeah. So that’s uh this is um one of the the kind of secret ingredients in this stew of maximum efficiency learning. I mean each of these things you know make sure a student has prerequisites in place before you ask them to learn the new thing. optimize the the the new thing into a sequence of steps that are just like very bite-size for the student to learn. Move the student uh make this like um minimum effective doses of of uh guided instruction and um and and and problem solving. Don’t make them do 30 problems of the same thing when you know like okay they just nailed the first two ready to move on. But at the same time give them more practice. this the space repetition um periodic quizzes all of this kind of combines together each of these are the multiplier on the on the learning efficiency and this uh knowledge graph topic like you got one from Matt right do you think like uh the same overall structure is transferable to like other domain I think like there’s uh somebody on the Twitter uh that is asking like um is there one for like graduate level uh physic computer science like um should be coming to build. What’s your take on that? So I think uh absolutely there are knowledge graphs or you know any subject that is hierarchical um like yeah computer science, physics, anything math adjacent like for sure is like a a really clear candidate for a knowledge graph. Now um this is actually on our you know we we just released a a math a mathematical methods for for physics course um and we are we have a a machine learning course that that is uh like almost almost ready. So it’s we are um we are actively expanding out our our knowledge graph be beyond math proper. Now how far can we expand it uh like how you know into biology or or history? I mean, some just get flatter and flatter kind of the the further you move from math, but um my my gut says you can, you know, take a there’s probably uh things that that transfer over uh to some degree. um like you I think it’s it’s kind of like in um you know in in in math uh the the hierarchy is kind of strict where it’s like you have to learn linear equations before you learn to solve a quadratic equation by factoring like there’s just no you have it’s it’s it’s it’s a practice subskll there’s no way around it now I think like in other subjects like history and you know there’s there’s sometimes there’s not necessarily one right answer to to like you have to learn this before that. But there may be ways that that work out nicely. Um I don’t know. I haven’t really thought a whole lot about like beyond um the the hierarchical subjects too much, but I know if subject is hierarchical, you can you can teach it with a knowledge graph. But I remember like uh when I was pushing my learning uh stuff um I was also looking at like literally other things and um I realized that there’s some subject that has very different shape like um uh math programming physic they all went into the same bucket right like of of tricks that I could use biology already there it’s like it was a bit different like uh like I remember like most of my stuff was compressing and summarization Right. Um, and then making like the stuff as vivid in my mind as possible. It was more of a modelization thing, right? I was using flashcard a lot for facts, right? And then I was trying to kind of bring it down to something I understood. It didn’t matter if like it was not actually the stuff, especially biochemistry because nothing makes sense. Like all of the protein have like threeletter and then like a bunch of Yeah. And then some of them are are funny. They’re like called Sonic Hedgehog, right? For some reason, right? That was very different. And there’s some things that like I remember my my wife was is into archaeology and like I was helping her into one like study in one of her class and was like I’m going to also study this stuff and it was like about uh Greek columns right and all of my trick was like okay what how do I do that how do I learn about like all these facts about these columns and like these dates and stuff and it was it was um much different like uh it was again mostly about summarization and bringing like the stuff into my world view. Um but that that was very hard like I had to like put more I would say thought into um bringing that material into my stuff compared to the other stuff where I could just like go and like do and while I was doing I was able to kind of like automatically subconsciously ingest the information you had mentioned um yeah just an experience of like learning biology like archaeology and like the different experience. So, I I uh last fall I started kind of dipping my toe into into learning um biology um trying to leverage uh these cognitive science principles uh mastery learning this this kind of stuff and just like kind of seeing what would what what does that even look like? And I don’t I don’t know that I’ve figured out the the most efficient way just yet, but um I guess I can I can tell you a little bit about what what I kind of stumbled into and I I’d be curious to to know like how that matches or does not match with your your own experience and what’s worked for you. Um, so it was kind of um, so with with biology, I I I took a similar approach of just like so my my my wife uh is doing a PhD in biology and so I a lot of that kind of comes down to uh, genetics. Um, and so I wanted to be able to, you know, talk to her about uh, that that kind of work and and not just have it be like a tutoring session where she’s like, “Oh, this is Justin. this is how genetics 101 like no I want to like actually be able to you know talk at a intelligent level in the in the conversation and and and kind of understand her work on a on a higher level. Um and so um and so a lot of this was just like you know it it kind of came down to understanding the kind of identified this model of like okay I need to understand what what exactly is a cell and what’s going on inside of it that allows it to function in the body and and and so a lot of this like you said was just compressing like there just a lot of stuff a lot of [ __ ] going on inside the cell right it’s like how do you and all these like things that the cell is is interacting with other cells or manufacturing proteins or whatever. It just it like a lot of it comes down to being like being automat understanding this is about like being automatic in in the process of it like what’s doing what who’s doing what where. And so I kind of turned this into into um like uh problems style. So like um just minimum effective dose of this part of the process of the cell and then like okay ask me questions about this like um like what happens um if if uh this happens in the in the ribosome while it’s producing a protein what ends up happening does the protein still get produced or is there failure or what you know things things like that a lot uh a lot of these kind of almost like the the equivalent of mathematical like problem solving questions. And so that that ended up um working pretty well and and got now it worked pretty well when I was doing it and then I kind of fell off the wagon. Have not done my space repetition. So I’m very rusty on it now. Um but uh at my at my peak I was I was actually able to like have a lot of like uh conversations with with my wife about how the you know just how you know the the various genetic variants of whatever are causing things to happen. and the virus is infecting the cell in this way launching this kind of attack. Oh, this screws up that mechanism inside the cell. That’s why it’s like that. That’s why this happens. Um, yeah. The one thing that I was not doing though, I’d be interested to to to hear what you think on this is I was um I I was learning this from an LLM and I was I was asking it to like it it started initially trying to ask me what are your interests like tell me more about this and that and I was just like I my interest is in learning this stuff as fast and efficiently as possible. I don’t care about like like don’t try to Exactly. It was like I was like cuz it was trying to like I knew what it was doing because this is like you’re like kind of what you kind of default to sometimes as a tutor is like well what are you interested in? Why don’t we make this relevant? And I’m like I already have it relevant. Like my I want to be able to talk to my wife about this stuff at a at a high level. Like let’s go just tell me stuff. I want this to just burn it into my brain. I’m not going to become a biologist. I don’t need to know like all the little things of like I I don’t want to explore the space. just tell me like how it goes, what works, and I want to capture this this automaticity as as as quickly as possible. And so I was periodically having to tell it like don’t make like I don’t want to hear like an like don’t ask me an don’t offer analogies unless I specifically ask for it because it’s kind of slowing down the experience. Yeah. I guess I’m I’m curious like when you were kind of uh getting an understanding of these these biological processes um how would you relate it to your own or like did you mention something about ma making it like uh relevant to to to your own like imagery or stuff? What did that look like when when you were doing it? And was that like an efficiency speed up or was it like a motivational thing or I’m I’m just curious to know more about it’s more it’s more than this like um um like a very very abstract concept are kind of hard for your brain to kind of memorize and grasp especially if they have no like relevance to to like the world around you, right? Like like and this makes stuff not stick at all, right? Like especially like a memory wise, right? Like most of your brain is like highly hard word for like like a space and time type of computation, right? Like if you if you were to think about it and if you were to close your eyes, you would be able to remember everything in your house, right? Like well where every little spot is and like where like stuff interact. And if something was different, you one day you like open up the door and then there’s like a furniture that was not there before. You will not be just like me and then you move on, right? You you already mapped this out and then it’s not now this thing is there and it should not be there and then it it like it goes into like this cascading of like why is it there? What what’s up? And then you start to look around and see stuff. Um so at the base level like uh since like this is more of a memorizationheavy type of situation where you have to like remember these facts that are not grounded in like the real world. I’m trying to bring them into like stuff that I can visualize and I I spend more time on the visualization aspect. Um so like but it doesn’t have to be like very very uh concrete. It just need to be make sense for me right? Um, so the ribosome for instance, like I was kind of almost looking at it as like a little factory and I was drawing it like this but with this aribism shape, right? And I was looking at the stuff that goes into like the peptide as um um as like some sort of coal based chain fuel of some kind, right? And I was looking it more into this mechanical sense. Uh then if you were to really look at it, it’s like is it just just a long chain that is folded within in specific configuration. If you think about it like this, no way I’m gonna make it, right? No, I was kind of um reducing the level abstraction to something I could like pinpoint in my head. Um and then I was doing this for all of the different um kind of category. It it didn’t like it didn’t have to make sense for anybody else than me, right? And some stuff like it was more abstract because I was okay with it. And some stuff was way less abstract because like yeah this dude need to like this threeletter name pro team with like four numbers after it need to be something. It need to be a dude right like and then I was making them like a little characteristic. Um but like one thing that really helped me out uh when I get into these space where you have to quickly get like the context and it’s not grounded into like clear boundaries right like the biology stuff is interesting because there’s no clear bound like we’re making these sort of boundaries but when you look into it it’s like yeah actually it’s like it’s more a bit wishy-washy like that right uh and when you look at the fact it’s like actually these three experiments says this but these two experiments says this other thing right I’m trying to around as much as possible the space first. So like, okay, what’s in the cell? Go, go, go. And then you look at everything that is in the cell, right? There’s nothing else. There’s not a like a fling flow that is in the cell. There’s there’s none of that. There’s like the 75 like bigger things that are in there. Fantastic. I’ve like closed the space a bit, right? And then I can I can start to sharpen my vision about like what they are, right? And how they interact afterward. And at the last layer, there’s this kind of layer of like we don’t really know. And uh like uh I’m trying to like know a lot about what we don’t really know on the stuff that I’m actively working on. The rest I’m going to put into the abstraction of like work roughly like this. That’s the flow I usually used. And this is also the flow I I use in neuroscientific uh stuff because like it’s the same thing like you have the cell but then they are in a population and then they’re in a region and then they’re in the substrate and then they they are getting like fed and like have to live and then they’re shooting out like these vesicles and stuff like all of this is way too complicated to take all of the little details at the same time. So you map the space and then you go into understanding each of the component and then you understand the interaction and you make it as dumb as uh and as vivid as you as you can and if you do that then it’s like it’s more easy to to discuss it can lead you astray. If your mental model is something and has some biases it like it can literally mess up how you’re you’re looking at it. And we see this some sometime with like in in biology and like neuroscience like we’re thinking about the model of the brain this way and actually like it’s not exactly like this for for instance like the the syninnapse and then the um uh the neurons like one of the model is that it’s shooting up the stuff uh uh the the vesicle right and then the vesicle are entering a specific post synaptic terminal um but in actuality like there are micro tubil from one neuron that can stick out to the other Right? So they’re kind of connected now. So your old idea of like how this works kind of changed now, right? It’s not the same. So you just have to be careful. Information at a lower level of scale that uh has kind of broke your abstraction a little bit. Yeah. But then like does it matter uh is another element. And then like can we keep it fuzzy for like 90% of the problem? Yes. Okay. We’ll keep it there. So there’s a lot of this going on. Um versus like the more hard type of clearcut like subject. you don’t have as much as this, right? You don’t have to kind of go into like uh these these weird setup where this we’re not 100% sure how it works, but here’s our model and you’re like you have to keep in mind that like it works in 95% of the cases, but 5% of the case you have to have to go deep deep deep in there and figure out like uh the the the missing thing or like the the thing that are we’re not sure. So yeah and in biology the issue is that like it’s everywhere like neuroscience like it’s it’s everywhere like the if you dive deep enough into any of the research like in any of the subject the it’s not like we know really well about the nucleus and then like there’s this the outer uh shell here and the the upper of GI is like we know 100% like we don’t know much and there’s like this big arrow bar in all of this different direction and uh it can really trip up um Yeah, you’re understanding because then you’re like what even are we looking at right now? Right. I remember there was one class that I took. It was like lipid too. I was I was learning about lipids. So, and um it was like the gold gi apparatus um and we had like a whole classes during the whole semester just about that stuff, right? and how they were imaging the the things and then it was taught by researcher which was super cool and they were showing us the research but then I knew like I’m at the edge like I’m asking question they’re like I don’t know because they literally don’t know and they can’t like they’re running the experiment and stuff but like there’s there’s a lot of stuff that we’re not understanding there and like this helped me a lot get the better grasp about these material because you’re you’re really really uh at the edge sometime at the edge is like it’s it’s not like a smooth frontier right It’s like it’s a bunch of holes uh a bit everywhere and as soon as you solve one there’s like six different holes that are somewhat bigger that pops up that connected to the other holes when you discuss to the researcher versus like some people that are just like teaching the stuff. You realize how much it affected them and how they’re teaching because they will be much more unsure about stuff and then they will tell you about the caveat, right? So you you tell like what what’s what’s going on versus a teacher that is looking at the curriculum for like these more like a soft science right not soft science but like soft certainty right uh science they’re going to tell you like it’s like this it’s like that it’s like that but the deeper you you you dig into the more you realize that we don’t know much. Yeah. So I just have to keep that into consideration right. Uh yeah definitely. Yeah it makes like Yeah. One of the nice things about um you know developing math curriculum and and you know mostly carries over to um computer science and and and and physics is that like things don’t really change. Um like in in math it’s just you know it’s evergreen like there are there’s new stuff that you find that builds on top of like in in directions that haven’t gone before but on the foundations generally like it’s it’s locked in place. But um yeah, I guess uh right in as you kind of go out into the fuzzier subjects, there’s these like just complete overhauls of mental models of what’s happening. And like I I remember like I was like doing consciousness research. There was this very famous experiment and then like it would publish and then they were doing all this stuff and then we like there were other people that tried to replicate it and they realized like it was not actually like 100% like solid. But by the time everybody like kind of built up their mental model on this stuff and now it’s like it’s not there anymore but it’s still there. It’s it’s um it’s a whole whole new world of confusion. Right. If we bring it back to to Matt I really like uh some of the allergy that you’re making with sport and I I I think like you’re also like you were also into Khalist. I never can say it calisthenics. Yeah. Yeah. Yes. And um you you make a lot of parallel uh uh within it. Do you think there’s something specific uh from from uh from that like uh stuff like sport coach uh can uh can uh learn from it into into learning like the the actual skills uh for their uh for their uh students? Yeah. you you know um one of my favorite uh I I think one of the most quotable figures in the in the study of talent development um John Wooden legendary basketball coach and he he he has so many quotes about uh sports uh that apply perfectly to to learning in general and so I think in in just any sort of skills based learning at least um mathematics, physics, coding where you actually you know you’re having to to perform manipulations on whether it’s objects of information or physical objects or whatever. um when you’re performing these these actions um and you’re having to build up to more and more and more complex actions um this this all kind of follows a similar suit where it’s like you know all these cognitive science principles of um master a learning like get your prerequisite actions in place before you do well a lot of these are kind of they’re actually more obvious in sports I think like you know let’s um before I often like to go to figure skating uh for sports analogies um because it’s just so dependent on underlying skills. So before we get you doing a uh skating backwards, let’s have you skating forwards. Let’s before we get you spinning on one leg, let’s get you standing on one leg. That’s the mastery learning. Let’s also um you know, spaced repetition. Um just cuz you were able to stand on one leg last week does not mean that you still um know how to do it. We have to continue practicing this. um things like layering, just layering on more advanced skills to lock in the fundamental skills. It’s like, well, you’re good at standing on one leg at balancing on one leg on your skate. um you you’ve gotten kind of your your your your ability at that has saturated because you’ve been practicing in an easy environment, but once you kind of like if somebody tries to push you a little bit or trying to like spin around on one leg, this is going to this is going to force you to develop robustness in the in the underlying skills. Um, same thing happens in math like in calcula. There’s a there’s a joke that like calculus is where students actually learn algebra because they are uh held account to account for applying a lot of the algebraic techniques. Um, yeah. And uh there’s interle um so doing instead of you know if you’re going going to do a skating practice um let’s not just practice stopping with your right leg the whole time. Let’s also practice stopping with your left leg and not let’s not just do stopping. Let’s do a whole bunch of other techniques. Um there’s um kind of retrieval practice even in sports. I think is this like well you think of it in in learning it’s like on a on a on a test or quiz you’re retrieving something from memory with no reference. You’re not getting a warm start to it by solving some easier problems leading up to it. You’re just like right on the dot under time constraints. You got to pull it out of your brain. Well in sports similar thing. You got to develop a a a level of uh you know, maybe you can you can do a a a spin on your on your figure skate if you practice a few times beforehand and then you can do the spin as you get your balance, but really you want to get to the point where you don’t need that practice leading up. Um you know, stuff like that. I think this all carries over really, really well. No, true. And I think like also like having the skills that you’re thinking is important because you shouldn’t think about them like as you’re doing the stuff like uh otherwise you’re going to use like mental load in order to concentrate and do the the movement that is needed but maybe the the the opportunity is already gone right or like you messed up in in this sequence. Yeah. Exactly. If you’re playing hockey and or basketball is a better analogy because if you’re having to think about how to dribble or how to pass your teammate or stuff like you just you can’t like you’re not going to be seeing all your teammates around like who to pass to. Yeah. If you Exactly. And and in my case I was um I was doing swimming, right? Um you get that also like when you’re practicing like you have to kind of understand all the movement and like it’s like there’s a lot of coordination about all the things that are happening at the same time. But when you’re in the meat and then you’re you’re going for it, like there’s no you shouldn’t be thinking about anything. If you’re thinking about something, you’re going to you’re going to mess it up. Uh for sure. I think we can like move the discussion back toward like the sports stuff because it’s uh a bit less abstract than math, but there’s a lot of similarity, right? There’s a lot of similarity about like how the learning is is done. And like um uh we’ve discussed a bit about like the the the sports coach analogy here where like uh there is some stuff that the that the learning of the mat happens that let’s say the coach could could use also it’s kind of a similar type of a phenomenon. Um but one thing I’ve realized like when I was um I was doing like competitive swimming, right? Also like when I was motivated and I knew what I wanted, it was re really easy like when I was training to understand like yeah like I’m trying to like do the 200 butterfly. So like I don’t care about the breaststroke stuff. It’s not that important. I want to get focused on this. So I need to like uh like really nail these five things, right? and I was watching recording of like the people at work hitting the time that I needed. So like there was all of this that uh like went into my mind. Um but one thing I realized is that like some of the best swimmer whenever they had to coach they were not that great. And then when you were looking at the great coach, they were not necessarily like uh like the Olympians and and stuff like that. They were actually a bit lower in um in the in the their performance. So, and this got me kind of interested like there seems to be a discrepancy between like doing the movement and being really good at it intuitively, right, without even thinking and then um understanding it enough to be able to explain it, right? Do you think that like there’s like a similar phenomenon in Matt that is happening underlying or or or not? Absolutely. Yeah, I I think so. So we I think we touched on this last time talking about how like you know in in college you can get a professor who’s like just widely recognized in their field and you get really excited and you’re like oh my god I’m going to learn so much and then they come and they’re just like proof they’re on proof on the chalkboard. You don’t even know what they look like cuz they don’t turn to to face you just like on the board and like they don’t really grade the quizzes or exam. There are no quizzes. Homework sets are graded. Did you do the homework? Okay, good. Good enough. let me get back to my research. Like, don’t bother me, kid. And like, yeah, so that can that could totally um yeah, I I think that totally holds. And and I think a lot of times, um the people who I mean, we’re we’re talking kind of about in in generalizations here, but often uh the the the the people who rise to the top of the discipline, um they’ve often gotten a lot of stuff for free, uh like a lot of like, you know, cognitive machinery or just life situation or whatever. There are things that they they they often did not have to figure out a solution for like you know you can think of like a mathematician who just like you know I never really had had challenges like writing my work down like it always like came naturally to what do you mean you you you struggled to write your work down you just do this or or stuff like you know I I never like these concept like what do you mean you can’t just read a a textbook and then it clicks for you and then you do the Oh, you just read the chapter, do the problems. What What do you mean the problem’s hard? Like that’s not like, you know, a lot of these people can like take bigger leaps in in generalization of things. They got like uh some some some like they got a GPU in their head basically, right? Instead of a CPU, they’re just running on some different machinery. Uh sometimes they’re all not necessarily different machinery but just more optimized in different ways like you know individual variation in working memory capacity and like retention rates and like there’s unfortunately this is a reality of of of learning that people are kind of like built with different specs like this and and and and yeah so a lot of like learning how to how to teach effectively uh comes from I think having to really push the limits of your own specs and make you kind like uh punch above your weight uh in terms of where you would otherwise be if if you Yeah. Anyway, that’s so I I would totally agree that this happens all the time. I think there’s there’s a lot of this, but also there’s a lot of like they don’t realize what they did. Yeah. So, like there there was one case um the guy was like a menace in the in the and they super nice dude, right? Very nice. Um I never seen somebody so confused and so confusing. He was not able to bring it down to the level that a student was able to understand it, right? He was always talking into his layer of abstraction. Um and like that’s kind of all he did for like 14 years and and stuff like that, right? The guy was smart, yes, but like I realized early on that like there was no way I could I could like uh I could like pass this class just with this dude. So I did everything else. I was doing the exercise. I was asking him for exercise and stuff. And one thing I realized that like he that’s also what he did before like it was not just some right like he was just drilling exercise again and again and then he pointed me to other uh uh part where the exercise were super good. Nowhere in the curriculum this stuff was was there. He kind of forgot that like you you have to ask the student to drill and they were not drilling it and while I was doing this I understood nothing right. So I was drilling the exercise trying to figure my way and then like by drilling them I kind of understood like what he was saying and then I could ask questions right and then he was like okay you understand I was like my dude right you you failed in like the metalarning aspect of the skill right you failed in understanding that these poor souls here right they don’t understand that you need to do that stuff so you have to teach that too right and if you don’t then you don’t guide them what they going to you going to say, “Hey, read this page on the like, okay, read it. What’s next?” Right? Like, they don’t know that skill yet. So, if you don’t also layer it out, there’s no way they’re going to understand like what they’re what you’re saying. And then you’re going to lose them for for real. And then as you move through like more complicated material and like more like more uh up thearchy, it just get worse and worse and then after at some point like these these skill are completely tuned out and then everything fails, right? There’s a few outlier that that succeed, but like um the rest fails. So I think yes there are some people that like have I don’t know like a better capacity to adjust material but the poor poor teacher in my view like most of the time they just fail at the basic of teaching right they actually absolutely the bar is on the floor they haven’t learned that skill and um I know I’m in this mode whenever I see a teacher start to kind of question the the mental capacity of a student right like hey like why you’re you’re like this back back in my days like this this this and And I’m like, “Yeah, man. You suck at this. This is the truth, man. You don’t you’re you’re you’re maybe good at the material, but the actual teaching, this is not it.” If year after year this is happening and you see sometimes with two class, right? It’s the same material and one class is not filling as much and the other one is like it’s a slaughter house. Um, yeah. And then you they don’t they don’t have the they don’t they haven’t learned like they just intuitively like did the right stuff or like they were taught the right stuff and then like yeah this is how it’s done but like they’re not aware that they’re completely lacking in this whole uh whole domain. Yeah. Well I actually two uh kind of funny like concrete stories of watching this play out. Well, one is very brief and it’s just um you know I I I saw differential equations class intro differential equations class uh being like the the the the guy I wanted to to to to teach the differential equation like introduction to it is like well the the the solution to a differential equation is just the kernel of the linear operator. That’s that’s all it is. This is like that the students have not done differential equations before. they have they have not like really gone over kernels and linear operators aside from maybe a little bit in linear algebra. So just trying to jump to the highest level of abstraction and the students are all just like what like so and so I was like I was actually uh tutoring a bit for for this class and like the instructor was so out of touch with like thinking like oh the students are like working on their their proofs they’re learning a lot and meanwhile everybody’s just like what is this stuff and so like I’m trying to like fill in all the gaps and like it’s just terrible. Another instance is uh you know I actually um I worked with my my uh sister-in-law a couple years ago and she’s taking real analysis and like I mean real analysis is the the stereotypical like math major record course right cuz that’s where it transitions from a lot of like you’re used to you know linear algebra multivariable cal differential equations it’s kind of concrete and like maybe taking a methods of proof course kind of dip your toe in the in the water of like abstract proof writing then real analysis comes and is like if you’re not like 100% solid on this proof writing, you’re going to struggle. And um and so the instructor would basically he he would serve up maybe like five homework problems um every week and those five homework problems were at the highest level of abstraction. So it’s like we’re not we’re going to skip over proving that a set is closed or open or stuff. I’m going to jump right into theorems of like prove this closure property of the nd dimensional sphere under these conditions and you’re like like man can we just like work in like one one dimension first with like some actual numbers not like just like you know stuff like that and so crazy part like the homework would would not even get like really graded. It was kind of like a for completion thing. So, it’s just if you attempt it, if it looked like you attempted it, then you get like you get points. And then there were there was a uh there there were no quizzes, there were no assessments, every um except for a midterm and the final. The midterm the midterm was given in the middle of the year and it was handed back I think a week before the final. So, there was no signal feedback loop at all. Yeah. There’s no feedback loop. Yeah. And so I’m I’m told that during class like the way he would run it is he was like write a map up on the board, write down the proof like, “Okay, you guys have any questions?” So he he would mistake the um the students silence for like he would say like, “Oh, great. Okay, this is you guys already know this. Okay, let’s move on.” Meanwhile, everybody’s like like we don’t even know what questions to ask. But otherwise, he was like a nice guy like you have said like Russian is very nice. They just are nice. just don’t understand what’s going on in the skill. But like the teaching it’s it’s so poor. I I had a class like this, right? And it was my final um year of undergrad and I was I took like a like a a very advanced um uh kind of um time series analysis signal processing course. It’s it was I knew it will be hard because it was like one of the worst like class and um it was it was uh given by um a chairman of bio engineering of Canada and stuff. uh big dude, right? The research on point and like I swear to God I I I couldn’t understand what he was saying, right? And like there was always at least in this case there was always practice every week. So we had the the practice. The feedback was a bit lagged in time, but like I could pester the TA and get the feedback that I needed. But in class understood nothing, right? At some point I was fed up. So I put myself first row straight in like uh on the on next to the chalkboard. And then every time he would do this, I would just raise my hand and said, “I understood literally nothing for the last three minutes that you started here to here.” Yeah. Nothing. Nothing. Like what are you saying? Right. Literally, this is what I was doing. And what happened was like the other people in the class were like, “Yeah, me too, man. Like what is this?” Right? And then this is at this point that like close to the end that they finally realized that like, “Okay, this is not working.” And it was taking us back. I had to force him to take us back. It’s very hard for a student to do this because like you there’s the peer pressure and then to like I don’t want to look like an idiot. But I don’t mind. I was there. I literally chosen I could have chosen any other course, right? I chosen that course because I wanted to understand the material and now you’re presenting this stuff to me that was not in like the textbook that we had, right? And I don’t get what you’re saying, right? Maybe it’s a notation problem. Maybe I’m just a dumbass, right? But I want you to walk it through me, right? to bring me to your level. Um, and actually the course got better. It got much better then. Uh, but it was like a ping pong every every week. It was uh it was me like in the front and like I don’t understand absolutely anything, man. Can can you tell? And then I was able to to get like show him like okay I get this because you said that but why is it different than this? Oh, I using another notation. And the whole class is like oh yeah of course like if you use another notation that is in the book like how can I understand this stuff? Like hey are you not always engineering? And I’m like, I’m a neuroscience dude, right? I don’t have an engineering background. And he’s like, oh, okay. Like, yeah, maybe like that will be a good thing to know before we go over there. Know who who you’re who you’re teaching it to. Yeah, absolutely. You know, another thing that strikes me about the how you like, you know, you’re sitting in the front row asking a bunch of questions. So, in addition to kind of like identifying these kind of missing prerequisites for, you know, a lot of these prerequisites shouldn’t even be prerequisites like a notation change or whatever. like why why would you do that? But I’m sure there’s other parts that are like legitimate missing prerequisites that were just kind of skimmed over that you’re like forcing them to backfill. I think another uh thing that that that probably made this approach work really well is that you’re kind of you’re you’re effectively simulating minimum effective doses of like instruction and and active you know you’re generate like what you’re thinking about it kind of answer trying to get your own question answered. It’s like a a I jack in the class. I pay for this. I pay for like I pay for this. I’m here. I’m going to get the material I need right now, right? And if I don’t need and there’s I’m not I’m not socializing. I I don’t care about the dude, right? Like great guy, right? Absolutely great guy. I don’t care about him personally, right? What I care about is getting my learning in. And if now it’s so easy, I might just skip the class and do the textbook and like be done with it, right? Um but uh yeah like at some point like it’s a bit sad but with some teacher you have no choice but to literally go and see them ask question and go see them in the office hour and just like get the knowledge out of them like forcefully almost. Well, this is like a broken model of reality how learning. You don’t teach a student by talking at them for an hour. Like that’s going to overload any and unless the student already knows the material already, like they’re going to be overloaded with like their working memory, their cognitive load. You got to be breaking this stuff up. like I mean filling in the prerequisites but also having students like you know like why would you talk at a student for for even like 10 minutes or more before having them like do like just do an exercise and you know some some classes do a kind of like a a very like try to move in this direction by doing like clicker exercises and stuff but but that’s uh I mean it’s only a baby step in this direction to what it needs to what it needs to be but yeah there’s something that I read on your blog that like um kind of uh go into the same line which is like automaticity in computation is a prerequisite for conceptual understanding right like not not a substitute for it but like a prerequisite walk us through your reasoning about why this so because I think it touched here and like could you think of ways of like not having this automaticity but still having the understanding is it even possible in your in your Okay so here’s I I I think this like I think a lot of people who would um you know clash with this or like if there’s a line drawn in the sand on this and there’s a battle between two sides. I think the point of contention it really boils down to what what does it mean to understand something like there’s so okay there’s a kind of conceptual like you know you you watch a video on say neural networks right you you don’t know really how they you’ve heard neural networks kind of underpin a lot of like modern AI and you’re like you’re interested in it and so you watch a video on on YouTube and you learn about back propagation and you learn about like regularization and some of these things and you know, train test set, whatever. And so you you get a little familiar with this stuff. Um, do you understand it? Like you definitely don’t have automaticity. You haven’t worked any problems in it. You haven’t set up any thing on your own. Do you understand? Well, there’s this some kind of I guess like you you’re familiar with some of the terms. I personally I would whenever I talk about understanding, I mean like understanding to the level of like conceptual understanding to to act to to the highest degree. conceptualizing like you know there there’s a difference between how maybe like a a basketball player conceptualizes strategy versus somebody who just watches basketball games and cuz the basketball player like knows like at the fundamental level these strategies are like work or don’t work because of like these movements that I’m doing like they’re when they’re thinking about these strategies and and stuff like that the parts of their their brain that are actually like activating like movements and and thing like It’s almost like simulating like a a simulation in their head of all the underlying things versus somebody who’s just seen this at a high level. And so I think I mean you can acquire maybe like um some familiarity with just the like okay back prop is something involves hardcore multivariable calculus and you somehow figure out what how to adjust the weights of the the network like I mean if you call that like conceptual understanding then yeah I I guess you can you can get that without having automaticity on on the um you know actually concrete uh computations. But if you talk about back prop in terms of like you know how do I optimize like say this issue is happening and like the neural network did this this is what the loss curve looks like and this is what layer three reads as like what’s wrong with it like you know that’s I I would say that’s a level of uh of conceptual understanding that you you can’t get unless you have actually worked through the computations. I agree here. I think like um uh there’s two things I think that u especially like on the machining side is that you need to have done the movement right and then like in my view you need to have like diagnosed enough of uh the the the the movement around the movement in order to be able to to like uh directly see it and say like oh you you look at the loss curve and you’re like this is your problem you should fix that and then like then you you you do it. This is like the the point where you’re you’re truly able to understand it, right? You’re able to solve problem not even with without even kind of just looking at the output you’re like this is what happened for sure. Yeah. I think it like in sport also you see that like uh like the I think the coach the the one that are great they are at this level. They’ve already done the movement. It’s not like the guy is NBA and he has no clue and he’s just watch videos that that’s not it. He already played. He knows the movement right? But then he knows also like how the movement comes together to get to the final result, right? And like I think like a pro alete maybe miss this part like they miss the how to get to whatever because they’re using the coach. They don’t have to think about this. They just need to go and do the stuff right. So they are very good but they not necessarily have like the full understanding of like yeah you have to go and then sleep like this, you have to improve like this. You have your diet need to be this stuff right? All of this need to like come around in order for you to get to like the actual actual goal. Um, but that level of understanding like you like if it was just a guy that like just watched a lot of stuff but never kind of conceptualize it in its own kind of own space and own movement there there’s very little that he can add unless like it’s like just a surface level like hey maybe maybe try this and then you’re like okay why I don’t know and then you move on the conversation. I’ve got a concrete example for for this again in the machine learning setting. Um so it comes from when I was uh teaching this um this really advanced um applied math and computer science uh elective sequence within math academyy’s original Pasadena program. And so we we got students from the point of you know they came in like having their their core engineering math in place. They’d learned linear algebra multivariable calc and everything. and and we scaffold them all the way up through like coding exercises, building machine learning models from scratch, no libraries. It’s like re-implementing um papers uh in in ’90s artificial intelligence. And one of the things that I remember just um that really stood out to me as like just validating this kind of like you have to learn the movements um in order to to to really understand what’s going on is when we were talking about um exploding gradients and vanishing gradients in neural networks. And so I I I remember having talked about uh this previously with with you you know some people who are who are interested in um machine learning but had not they didn’t have the really the math foundations. they had not gone through the um the computations and it it um it was always like a kind of like a fuzzy area where it’s like wait what do you mean why why do they explode or vanish like okay I guess the activation function the derivative of the activation function if it if it the slope is too high or it’s too low or is it if it’s unbounded or like there just kind of like you know something about the activation function then it’s like it’s bad and then like bad stuff happens. All right, I understand it. Let’s move on. Let’s move to the cool stuff. But with with these kids, they they I made them work out um some some simplified scenarios um or like small like maybe a a two three layer neural net of like two nodes in in each layer with like some nice numbers. So the the computation took like maybe like 3 4 minutes or or or whatever. But they could see very clearly the effect of different activation functions. And so I I would like okay what if we use let’s just say we used an exponential function as the activator or or just even a a a linear function with with uh you know um or a quadratic or something where the the derivative is is continually increasing and then they would see in their in their computations like wait when I chain roll it and I take the derivative of this thing and then I multiply it it just keeps getting bigger and bigger. What do you mean like why are you why did you give me a bad Justin you gave me a bad activation function. why did you do that? And then I was like, ah, all right, you see the problem. Why don’t I give you an activation problem that doesn’t have that property and then it’ll just go go flat. And they’re like, okay, great. Thanks. And then I and then they they do that. They work the computation. They’re wait a second. They they just get one round in like and they already anticipate how it’s going to play out. They’re like, wait, this thing is already like basically the derivative is is saturating. It’s it’s gone to to to zero and uh and and and and it’s just all all the terms are going to go to zero. Like I don’t even have to work out the rest. I already know how it’s going to play out cuz you just chain rule multiply chain rule multiply goes to zero goes to zero and then at that point like they they get it. They they get the exploding and vanishing like in their bones like like some some researcher that are super good um they they feel it right. they they already kind of uh feel it and um it’s very important in research in my view because like some of the stuff is not quote unquote true, right? Like the the ground is is there but like burly, right? So some stuff that we we it’s because of this. It’s like yeah but there’s like four asterisks around it and when you actually like feel the stuff and like understand it you know where what you can try and where you can move and then you can get like some really cool result after that. Some are super empirical because you use like that kind of like empirical based uh knowledge but the true understanding come here in my view like it’s like you do the stuff and you’re not able to explain it. Yeah. Okay. let’s like you can build up the theory like like a do it in a systematic way in this case and then then you’re able to explain what’s going on. Um but like like empirically you can you can feel which of the direction um to go. I wanted to move on to like the a funny question about um confusion in learning, right? Do you see any upside of confusion like in learning like just any any of of them? Because I say that because in my like um own research and professional work whenever I made like a very good breakthrough and then like it was kind of rewarding in some shape or form. Um usually like I had guidance like I know where I wanted to go like I know like roughly speaking like what was going on. Um, but I had to sit in confusion, right? And like kind of be just like okay with being there for an extended period of time while I figure out my stuff, right? But like that was something I had to do. But like the best stuff that I I produced was when I was in this situation. What’s your what’s your take on that? So I I I think I think the experience can can um vary a lot depending on whether you’re kind of like scaling up on, you know, known knowledge that’s just like, you know, this is figure it out, like just learn it versus kind of at the edge of the frontier where things are getting really fuzzy and like what’s even true and stuff like that. So I I and I I think what the confusion represents like that, you know, that the moment where it clicks in place and you just have that insight and you’re oh my goodness, this is this is it. and and then it clicks into place. So I think when you are going through the you know the knowledge graph of just stuff that that that that is known uh this is foundational material. I mean I I I think what that amounts to is really like you you you are missing some prerequisites and you build up all like maybe there’s like um this topic sits a bunch sits a top maybe like uh five different prerequisites and and uh or maybe say like 50 100 ancestors uh ancestor topics prerequisites of prerequisites and stuff and there’s some there’s one learning path in there that you haven’t really, you know, it’s just missing. You haven’t filled out. And see, you got most of it filled out, but it’s there’s one pillar that’s kind of incomplete and you’re just you’re banging your head on the top. You know, why what what is wrong? And then at some point, like you kind of whether you kind of stumble into it or somebody directs you to it or whatever, you kind of realize, oh my good, wait, this pill that’s the oh, so this works that way and that’s how it works. And then it kind of clicks more into place. you kind of like uh fill in that that missing that missing pillar and it feels amazing, right? It’s like the click. It’s the aha moment. But when you’re in that foundational body of knowledge, it’s like, well, there’s a lot of time that you had kind of wasted being confused that like if you just had that pillar in place beforehand, you wouldn’t have had to like sit there for for hours or or days just ruminating over this thing. And you would had the pillar. would have you wouldn’t have needed you wouldn’t have experienced a click an aha moment because it would have just been like wait what’s the big deal we just put these prerequisites together like what what’s hard man like that that’d be it and so so I I think when you’re in the foundational um the like body of knowledge the the these aha moments are actually something you you want to try to avoid. I mean, if you’re in a well structured curriculum, you you won’t actually have I know it sounds like a little counterintuitive, but um it’s it’s like I remember somebody we we actually had a uh at at at Math Academy, uh we were talking to this was years ago, there’s a uh somebody we were talking to about like um recommendations for for growth on on on social media and stuff and and they had recommended something like, well, you got to give it like a like have a nice name to these like aha moments or snap moments and like and and like talk about these things and I was just think like wait that’s like the opposite we try to smooth it out like so you don’t have that like you’re climbing out of the hole and you’re like I made it I made it like no they just make it smooth now the thing um eventually you run out of this knowledge that’s known right and you get to the the end uh of the the frontier where things are fuzzy and that’s kind of I I think like you do I think this is more what you’re talking about where it’s like you’re kind of in a setting where this has not been fully mapped out. You’re going to run into situations where you just have uh missing prerequisites because everybody has missing prerequisites because nobody knows what are the prerequisites. That’s that’s kind of what we’re trying to figure out. We’re trying to map out the the how to extend this body of knowledge further. And so part of this mapping it out means like identifying what reframes or prerequisites are are necessary. And I think as a researcher I that that’s that sounds like it would be a very valuable moment where you know you’re kind of sitting in confusion. Why am I confused? It’s because there’s some missing prerequisite. Holy crap. If I figure out what this is, resolve this. Everybody around me is confused, too. If I can figure this out, that’s going to be a high value contribution. So yeah. Yeah. I that’s that’s my take on it in this in this setup. Um, like I I I learned to kind of love the the conf like when I see there’s like no confusion, I’m like I actually don’t need to be there. Like somebody else could take my place here and then like do the stuff. We’re we’re good to go here. But when I see like some area where there’s like a lot of confusion, I I I find like there’s an opportunity to kind of like do something great, right? That kind of push the boundaries. So I learned to love these. But like what happened was um uh I was you I was uh like doing the wrong type of confusion in in school by taking like class out of order right u because I was trying to get like get this type of I don’t know why I did that like this type of skill set a bit of like confusion bending or something like that where I was okay with like not understanding and everybody else actually understood and then like trying to like uh figure my way out of this and then like getting the prerequisite figuring out like what I was missing and then like being able to to do it with the time pressure because I literally put my grade on the line here. So that was that was okay and I I I agree like you shouldn’t do that like there’s uh if like there’s better ways of doing it when I was doing research as undergrad it was much better than when I was doing that because there was no point in doing this uh for for the outcome. It was more for like practicing the skill. Uh but like when I do I was in a research setting um or uh also in um a startup setting like there’s a lot of confusion there too right there’s no one that going to tell you like oh you want to build this company well good news and then here’s the steps like there’s none of it like you can take a look at what other are doing but this is fuzzy because is he figing it like can I get access to this financial I’m not sure it’s working right does he have the same kind of population as me not really right is the stack the same no okay so I’m not sure how to build this. So, there’s a lot of that, too. And then you have to kind of get there, right, while like like uh being around these confusing signal um throughout. But when it clicks and when you figure out how to make it click kind of click quickly or like reduce the confusion so that like you you kind of restrict the path of the nodes that you need to kind of follow. It’s uh uh there’s a lot of gains uh over there. So, like this I think you’re right. like that’s that part is is useful. Uh but like if if there was a known pet and you’re like baiting confusion just for the fun of it, there’s better ways there’s better ways to to to use your time. Yeah. It’s like generally doing hard things is is is good, right? It’s good to push yourself to do hard things, but also like when things are they don’t need to be hard, like don’t don’t make why why why make it harder? Like you can you’ll get to the hard things that really matter. So spend your time spend your confusion on things that when resolving it’s like spend your confusion on Yeah. on things that have high ROI on that confusion. Like if you if you’re confused, you get an aha moment, a click moment. Like how about make it something that actually like is a contribution to a field instead of just like oh this is a result in this math course that I skipped like no okay nobody cares 90 days on it for yeah for for not that much. you you hone that skill to be okay with the confusion but like other than that it’s like more of the meta learning uh learning happening here. If we switch gear to um learning meta as an adult because everybody has like this nice uh kind of curve of like building the stuff. So they are maybe in a I already in a position where they need to use the math and then they kind of understand it but a bit barely. Um so I got a lot of question about that. There’s uh the first one here that I ice shady on Twitter asked, “I’m an adult learning calculus for the first time. How do I rebuild the high school foundation while still moving forward through new material without spending a year going back to basics?” Um, so we actually uh we we we came up with the solution to to to this problem at at at math academy because we we were getting like a lot of you know adult learners asking this qu like basically immediately as we kind of emerged on the the scene on on on X Twitter um like got a lot of adult learners are all asking like the same question is like I what do I do? Do I I I need to kind of start over? I’m like like I’m not even at calculus. I I took pre-calculus in high school and I forgot like all of it. Like I should I just start with fractions like fourth grade math? What what do I what do I what do I do? And and so our answer to that was basically like why don’t we design a a a sequence that is the it you know we’ll we’ll start it with fractions. So you can go down as as far low as you need. you can place into the sequence, you can place out of material you already know. Um, but it’s going to strip away um all the stuff that is not that that is there for you know like in schools uh school common core standards but does not actually really get leveraged in in university math. So just to name an example um in geometry there’s this like inscribed circle theorems uh where you’re like okay I got a circle I got an angle in the circle and like what’s the if the angle li the the the vertex lies on the circumference of the the circle what’s the the measurement if you know the the arc of the intercepted by this thing it’s one of those things it’s like okay maybe it’ll come up a little bit if you do like uh the physics of of um like lenses and stuff. But aside from that, like you’re just never going to see this kind of thing. Not relevant to machine learning, not relevant to most physics, not not relevant to like basically any anything else. And and so there’s a ton of these topics that just they’re not very foundational for for for high school math. I think um I think it was maybe like a a quarter or a third of of the traditional school sequence. You can you can just kind of like strip away and it’s really it’s not a big deal. Um and you but you get your foundations, the really important stuff that underpins all um like a lot of the the serious math and and and physics and computer science, machine learning, whatever people want to do. And so we we we made this three course sequence the call it the mathematical foundation sequence and and our our director of curriculum Alex Smith was really the the one who put all this together and and and made it happen. Um but the sequence is um all together it’s it this three course sequence you you you you take the three courses and you emerge on on the other side ready for for for for university math uh like linear algebra multivariable calculus or or a a composite course like math for machine learning that stitches stuff together and to go from um this math foundation sequence the the very bottom to the very top. So assuming that you like really like you forgot how to add fractions like you are starting from almost zero um to to fill it out all the way um takes about 15,000 minutes. Now it might seem like a lot but if you actually do like okay how long is that actually per per day? It’s like well if you just do every every weekday for an hour for a year you can fill in all your foundations. um if you’re starting from the very bottom. And now if you if you decide to do like I’m gonna do two hours per weekday, guess what? Only half a year. Uh three hours per weekday. Maybe you want to split that up into like a morning session, an evening session, or a couple times a day. But um if you get like really really serious, like guess what? Like just a few months. Um and that’s assuming that you’re starting from the very bottom. Most people don’t start like just from adding fractions. They remember some algebra at least like how to solve a linear equation. So I guess like that. So really you can if you’re serious about this, you can get it done in uh in several months. Um yeah, the the the path is there. And we’ve had people do this and and come out the other side. You know, actually the the person who I referred to earlier who had uh signed up um math academy courses because he wanted to read this robotics book and do robotics and he just had no idea like way above him. He went through that sequence and he came out the other end and it was just world changing for him. Yeah, this is good. Yeah. Yeah. Um and like true like you don’t have to like hop around like the like the extra stuff um which is not directly useful for for your goals. So it kind of striped away to the core of it. I really like that. There’s a uh two other question from um Nick Cube and Zed on Twitter that says, “I’m doing the financial and learning how to do all this cool stuff, but I’m not really sure what it’s all used for. How do I fill in that motivation gap?” I think that it ties up to one thing that we discussed earlier. Um yeah, what do you think here? Yeah. Yeah. So I um definitely ties into what we were talking about earlier and like the the end of the pipeline where where I mentioned that we were talking um talking about having projects uh as just really cool projects as as kind of this anchoring goal for for learners where you know you can look and be like like wait like I I just take these math courses and and then I I I I get to like training a convolutional neural network to classify digits like what really or or like even cooler applications like uh like use a a neural net to um like approximate some some lengthy physics computation and run it really fast or or like use a like um clustering of like of of language symbols and like a a longlost civilization whatever like we got a ton of of of different um projects like that coming out in our in our upcoming machine learning course. Um, and we want to kind of extend this uh to the to the whole curriculum. Um, not just machine learning, but like actually do it in like just a you you know your routine calculus course. like why not have calculus with a bunch of really cool projects that involve like you know um and be integrated with a little a little physics and have one of the projects be like figuring out the viability of some uh situation for for for setting up a colony on Mars or something or like um figuring out like what fuel type of fuel is best for a a a turbo boosted race car or figuring out like what ingredients are are going to work or are not going work for like a certain um recipe given like chemical properties, what whatever stuff like that. Um a goal is that like if if we can cover the space with enough really really cool stuff and then like allow people to you know anchor on on particular projects that they want to do. Um and hopefully like not not just the highest level projects but also intermediate projects leading up to there like have some projects in algebra have some projects in calculus have some projects in um like linear algebra multivariable calculus math for machine learning all this sort of stuff there’s kind of like a a real like an application um that that seems really really interesting. There’s some really cool application that like they don’t require that much like like even just reproducing a paper like a result some of the time it’s like it’s not that much math like u I was reading a paper uh recently about um u uh trying to do maximum likelihood approximation but for reinforcement learning and like if you look at the appendix the the the the proof that they have there’s nothing complicated over there there’s literally nothing you go step by step there’s some debate basic stuff to you can you can see and like you just need to understand like the flow of the overall paper but you can reproduce the result directly and you can reproduce the derivation and explain every step of the way um just some basic calculus and you’re good to go um a lot of the stuff that are very approachable but like people don’t know like I said like the boundary of the knowledge so they’re like for sure I’m missing something but when you actually look at the stuff you’re like that’s it this is this is the only this only thing one paper that we actually are is going to form the basis of of one of the machine learning uh projects uh which uh remember I I I mentioned that I had a these high school when I was teaching this um advanced uh machine learning track within math academyy’s original school program I had the the kids uh reproduce um papers and there was one paper uh back in the ‘9s artificial intelligence like you say it’s like it’s a cool result and it’s surprisingly like you know if you it it’s um evolving neural networks to to play games like tic-tac-toe or or checkers or whatever. And so like you actually don’t even need to do back propagation because it’s an evolutionary procedure. And so if you can just forward propagate and and I mean that’s really just the the hard part is not not that much math involved. Um and then then you can do it. So, so yeah, I I think um you know h having paper uh reproductions is going to be a really cool thing as well. And I’m sure there’s tons tons of those papers. Yeah. Um there’s another question by Muhammad Ax that ask how should teaching math for kids or adult balance exploration and creativity with what can feel like a repetitive drill. He said like I remember reading a metician lemons. I think I also read that and feeling it should be all curiosity and creativity but also no I couldn’t have thought deeply without doing the boring problem first. I think you answered almost his own question. What’s your take on this? I think it’s one of those things where it’s um you know as as you’re kind of going we we’ve made this distinction between when you’re in the knowledge graph the known knowledge graph versus when you’re building on top of the knowledge gra. like at the frontier of things and so I think um I mean just from a pure efficiency standpoint like you know when you’re going through the known knowledge graph I mean you any sort of creative thing that you will do it unfortunately it’s it’s not actually that creative because it’s going to be rendered trivial by a lot of results. um like you could study the property the statistical properties of of random walks as as a as a you know as a as a solo creative project or stuff but like guess what that’s already figured out like so whatever whatever result you’re going to have on on that is probably known already now not to say that there is zero value to this because there’s motivational value to it right like if like if if the choices are like either I’m going to lock in If you’re going to like draw a dichotomy for yourself, it’s like, okay, either go monk mode for like all my studies until I reach the frontier and only then will I allow myself uh like any sort of creative exploration. Um like that’s like just mechanically that’s going to be the most um efficient way to get the frontier. But if you’re going to if that’s going to force you off the rails where you’re just like, man, I quit. I’m done. Like well then like okay, like just do do what you need to do to stay on the rails. And if that involves like just going down some some rabbit hole or whatever, just to like have some kind of like fun with it, have like experience like some some tastes of the rewards that that you want to like get once once you get to the the frontier of it, then like okay, do that. Do do what you need to do um to to to make it fun. Um, and everyone’s going to be different in in in this capacity and like how how much of that they they they feel like they they want to need or what type of rabbit holes they want to go down or whatever. Um but I I I think the real the the the point where this like you know the the exploration and everything becomes very valuable like a high ROI sort of thing is really once you reach the frontier that’s totally totally agree like um there’s a bunch of actually undergrad that I’ve met uh found really cool machine learning result at the frontier um and like the the because like machine learning also is very empirical in nature right like yes there’s all a theory but if you look at the frontier enough there’s a lot of people like messing around and just like trying stuff up right and you have this feedback loop about like did it work yes okay this is good positive thing so what I’ve seen a lot of of them um like the the very bright one do is that like they go into the frontier and they look at the frontier and they understand jack [ __ ] literally nothing right and then they drill and then like intensely and then they do stuff like um at the knowledge gap that they have and then they kind of build this up, right? But time to time they go back to this and then they tinker with it, right? So it’s not like tinkering with like a known thing like they for this they completely eliminated from their kind of regime. They go and tinker in the frontier and try to see stuff like and then but then they learn a bunch of things they have and then they’re able to kind of reapply it and get ideas and then they they unlocks a lot of creativity. Yeah, exactly. And like some of them like they’re doing some wild stuff like they they try things that like they have no biases. So like they’re like yeah why not this why why not this other thing and then they they try a bunch of stuff and then they find some cool surface or like a way to simplify uh the stuff and then they get good result that they can then like iterate with. But when they go back into their kind of learning, they’re anchoring onto onto this goal and they’re they they have this kind of desire to up their skill so that they can kind of go back to this nice project that is at the frontier and um and do stuff with it. I like the best student that I I’ve I’ve seen were doing a mix of that. Like uh they they were doing both the frontier stuff to for their own motivation and their curiosity. Um and then they were never bored into the basic because they’re like yeah I know I need that because I don’t understand this this part absolutely no clue what’s going on. So I better like skill up fast so I can get that uh and and do that uh fun thing. So this approach of like you know doing a like go to some kind of interesting project at the frontier and do what you can in it and then like backfill to to try to unlock more creativity uh or more just um ideas or whatever to to to to do with this thing or to understand it better. Um, I I I I totally see how how this can can work for for people who have um, you know, some some amount of like a a large chunk of foundational knowledge already in place. And it’s I I’m just it I think it kind of depends on somebody’s the distance between what they’re trying to do and what they’re doing because like you can there’s there’s other this may seem like a caricature but this this this happens a lot where there’s some somebody who doesn’t even know how to solve a linear equation and and they want to do like wow this AI stuff is cool I want how do I get invol how do I do something it and it’s like well I mean you can I guess you can like download and and run like a transformer former model or or something. Um, but if if the gap is that big like when it comes into like like back filling prerequisites, you’re going to be like, well, okay, I need to learn like back propagation. What is that? That’s squiggly symbol. Oh, is that calcul? You’re kind of like, so it’s like if if the gap to fill in is not very very big, then then then you can like you can be efficient in in terms of back filling like I just need this that that learn learn. Okay, new unlock. But if the gap is like gigantic where it’s like you you’re essentially missing like all of algebra, all of calculus and and that then I I I think it gets to a a point where it it can be in inefficient to try to like trace your way all the way back down to like fractions or algebra. Doesn’t know how to count. Yeah, I got to level and he’s like, well, guess what? you’re you’re going to need to learn this stuff anyway. It’s it’s not like you can just take like oh there’s like there’s there’s three secret topics in in all of like 300 algebra topics that you need to learn to unlock machine learning. It’s like no you got to learn ba basically all of all of Alistster like it’s kind of foundational and so there’s I think for these foundational like when a body is foundational like that I think it’s you know if you try to just like go and back you’re going to be ultimately just kind of trying to construct your own algebra graph and and this is a known thing this is this is if you just like take the alge take an algebra course uh you can get through it like like a lot a lot faster but but I I agree as you like reach like closer to, you know, like, okay, you’re kind of within striking distance of this thing and you can kind of see the full the full path between the project that you’re doing and and the map that you need that the, you know, the back filling strategy can it becomes less inefficient. Yeah, there’s there’s a lot of this for like most people, but I think also like the the folks that I I see even young that are um very comfortable doing that stuff, they understand really well like the metal learning skill like they kind of um already uh they know that like ah I can’t just like watch lectures, right? I actually have to drill problems and I’m so far down that like yeah I can look at this try a bunch of stuff but like I can’t spend like 6 hours a day here. It will be useless. I need to quickly go and like build these skills. So then they go and they see like the most like efficient pattern and they go through it as fast as humanly possible and they just kind of blitz through it. So like this is something that I’ve seen is like even if the gap is big in these people, it kind of doesn’t matter because like they will be able to like regulate what they’re doing, right? Like the goal is over there. I know I’m missing a ton of stuff, but this is what I want to do. And I tinker a bit with it. I’m like, “Wow, this so this is so cool.” But then when they go back and they actually try to like for real do it and not just like have this aspirational goal that is not attainable, they work on the right stuff because they know how to get like the the learning they need in order to get there. Um people that like kind of flail in in this. I see also a lot of people like this because they want to do the the stuff but like they have no concept of really how to to to get the the prerequisite knowledge in order to do this thing. uh they go into weird path like uh and they kind of get lost within themsel and like even if you try to nudge them back I don’t know like they have like some learning psychosis or something like they they’re they’re completely spread out and then they have the most monstrous kind of setup uh imaginable um and most of it is just wasted time like it’s just wasted uh it’s hard to bring them back after that so there’s yeah I agree like there’s like this this kind of discrepancy um yeah definitely yeah that’s I mean that’s a I I think That’s that’s right. There’s also the variable of like how how much does this person understand how to learn effectively? Yeah. That that plays the same role as like a gap in knowledge. It’s Yeah. Um last question for the adult uh learning stuff. There’s two people actually and Clifton that asks, “Is it ever too late to start learning?” Seriously, I know it’s not, but give me a compelling answer with example for reassurance. Let’s do a bit of a of a psychology here with the with the guy. Yeah. Yeah. I I have I actually have a a very concrete example. But before I say the example, I will say like there have been a number of um adult students who I’ve I’ve seen go from like even in their 30s like just go from, you know, I don’t really I I want to get involved in some kind of like cutting edge thing or maybe not even a cutting edge like I just want to become a software engineer. and and and they they start this um in their 30s or even their 40s or I’m sure there’s people who do it even later in life. Um if you’re willing to put in the the work, it’s I mean it’s the same amount of work. It the amount of work does not change whether you you you start this at the age of 15 and like emerge where you want to be at the age of 20 or 25 or whatever. like this it’s the same amount of work if you started at 25 or if you started at 35 or or whatever. And some of the things that makes it um more more challenging uh for well I think sometimes uh people think like oh it’s it’s it’s too late for me to to do this kind of thing. And I I I I think that tends to be more of just like I’m I’m not willing to put in the work given given that I have less free time now. Right. you go through life, you got like a family, you got kids, you got more responsibility. It’s just harder to to make time to do things aside from especially if you have a job too, like a a separate job. It’s like where where’s the time in the day to to learn? When you were a teenager, you could just like you just come home from school like what what are you doing like play video games or like go to the basketball court with friends? Like you know, you got all the time in the world to to do this stuff. But the amount of work does not change even if your your your time to do it does. But the time that you devote to doing the work is is within your control. You can choose to be more efficient in in in various areas of your life to to to open up time and it’s not going to be easy and there’s always going to be like lots of constraints and and stuff and you’re probably not going to open up as much time as as you did when when you were younger. if you but the the the the takeaway is if you put in the work you will get to yeah the the result if you find some way to put in that work and so I’ve seen this happen in particular there was um there back when I was uh working in data science there was a u I mean I I started out as a as an intern for like 6 months before converting to um just full-time uh and and so there was another guy who was like who was actually interning uh with me and he was I think he was actually in in his 30s. Um yeah, somewhere in his 30s or maybe 30 years old on on the dime side. So he had a um I his his story was was kind of like that he you know in in in high school he didn’t really he wasn’t really paying attention to what he wanted to really get out of out of like career-wise. He was he was more concerned about like I just want to have like kids have a family. Um and like that was his main goal and that’s a fine main goal. And so he did that. He accomplished that, had kids, had a family, and he was working um as a as as um call receiver at at a at a emergency center, like the if you 911, who answers the phone, him. Um and and so um he he wanted to get into like just you know a kind of career that would that would offer more um money and better just uh like life balance and stuff and just so could like give more to his family and and and stuff like that. So so he was like you know um like I want to go into software engineering now. I want to get into like data science. And so he’s and so this was um you know he was like in his late 20s when he even like this he started even thinking about this and and he he didn’t know like any math no coding background like none none none of this stuff and so he just signed up at his community college and um in in his late 20s like one of the oldest students there um and he’s like whatever man like I like this is this I got to learn the stuff. So, I’m going to learn the stuff like so and he’s like and at the at the same time he’s like he’s got his job that he’s like holding down as well. He’s got his family and stuff. So, he’s just this is not an easy like sort of like he he’s pressed for time. Um and and but he’s making it happen. Um and and so he he goes there he gets his computer science degree um from community college and then uh gets an internship at at this company and uh and and so he and I interned in the the same around the same time and he also uh converted to full-time as well and he he took it very seriously and and and really got good at at at the the kinds of um software engineering they were doing like and data flows and stuff. got the nickname like the lord of the data flow over there because he was like just he was so like into it. Yeah. And he’s like because I remember talking to him and he’s like man I really need this to work like there’s no but it’s like he was at a point where it’s like there was no what’s the alternative plan. He wasn’t thinking about like alternative plans. He was like no this is what I’m going to make happen. There’s there’s not there’s no hedging of like like oh if if this doesn’t work out I I guess I’ll do this or whatever. I I’m sure he would. There would be plan B’s, but he’s not thinking about he’s focusing on the thing in front of him, making it happen. And um yeah and and you know one thing that I um that I that that I think is true is like initially the the hard this journey like especially if you’re embarking on it later in life the the hardest part is kind of the the upskilling phase when you’re you’re you’re trying to prepare yourself to you know to to to get paid to do something that’s more aligned with what you want to do cuz at at that moment like you’re not good enough to get paid for it yet. So you have to probably you have to be doing a different thing to get paid. Um and that’s gonna eat up a lot of your time. So you have to spend time doing this other thing and spend time preparing for this new thing. It’s like where you’re you’re just pressed for time. Once once you can make that transition to like actually earn a living from the thing that’s in the direction that you want to do, it’s like a just total phase change because now you have like maybe eight hours of your life every weekday that you’re actually doing the thing that’s closer to what you want to do, improving in that direction and it’s it’s like you get a rocket thruster. And if you can just make it to that point, then the the start is difficult, right? like u you don’t have enough time and like if you’re going into the wrong stuff and like learning it wrong you’re just going to eat up your time and then like the motivation will will start to uh to falter a bit. Um and like uh also like for adults generally like there’s a bit less neuroplasticity um uh in compared to like kids right like adolescent there’s just generally less neuroplasticity uh but it’s not uh it’s kind of counterintuitive because it’s not a set thing like you can get more neuroplasticity right like you can make your brain more plastic if you learn like new stuff right and if you exercise also but like if you learn new stuff your brain becomes a bit more plastic you of neurogenesis that is like start kick in and you’re able to kind of learn better. So, but this first like hill that you’re climbing and trying to learn the new stuff but it’s hard and then you don’t have time. If you like push through it and then you’re you’re able to do it well, every day afterward gets a bit better because like you learn new stuff now you have to learn more new stuff but your brain is better at learning new stuff now and then you kind of snowball into that. And if the motivation is is in the right direction, you actually see that you’re moving, then it gets better and better. Uh but the the first first few steps are the the most critical here. Yeah. It’s like it’s kind of like you let life oify around you. You let the concrete harden. You got to break the concrete. Yes. But once the concrete is broken, like it’s a lot easier. This reminds me of uh you just one more concrete example. There was a a student that we had um uh well that we that we have and this started uh last year. Uh and he he was talking about how um you know he he he posted some some video talking about like the experience of like just getting a a habit in place of doing math every day. And he he was talking about how like when he started he just was not really into it. Like ma math is like I I think he wants to go into software engineering or or uh data science some something in that direction in the tech direction. Um but and if you view viewed kind of math as something like okay I just I got to learn this material. Um all right he he was doing a a degree in computer science. Um and there was like there was a bunch of math courses that he was just like man I don’t I can’t I can’t do it. I got to like fill in my foundations. And so he he started on this and initially he he was saying like man like every day I wake up and my brain just tells me like don’t do it man don’t do it. don’t don’t start working on the problems. This is not going to be fun. This is you should run away from this. And so it’s like every day and he just like there was at the beginning there’s just a lot of like you know just like shut up brain. I’m doing this and we’re going to do it like in the morning before you’re really like awake enough. I’m going to I’m going to beat you to like before you start yelling at me that I don’t like this. We’re just already going to be doing this. And and so but he said that like this continued for for weeks. It was like I I can’t remember if he said like 30 or 60 days or something like that. And he was like man he was like I I just accepted that this feeling is probably just never going to go away but I’m going to get through it anyway. And then like a week after he he just accepted this like I’m just this is what we’re doing. He he said like his his it’s he started like just not having all those those bad thoughts anymore about them trying to dissuade him from them. It’s like he’d wake up and he’d be waiting for his brain to be like don’t do it man don’t do it. Don’t you dare start us on math problems. And it’s like the brain like it just given in. It was like just like okay yeah I guess this is what we’re doing so like fine. Like it’s like if you get yourself you just go through the habit enough to get your your your brain to just accept that the that that this is happening. This is the the path of least resistance is really like you’re going to be doing the stuff I would dig in my heels like but this is like pure energy conservation from the brain right because it is using a lot of energy right compared to the rest of the organs. So like anything it can not do it will it will [ __ ] do it right? So if he sees that you’re doing something novel, especially later in life, it’s like dude like I’m where like I have to like do all this effort and at some point it just able to reconfigure itself to kind of lose you lose a bit less energy, right? And it has to to spend a bit more. But now it’s better at like doing that stuff. But if you keep doing this actually like your brain is just more flexible in general because you’re not like most of the time you’re not in a in a resource scarce environment where like you won’t have enough to eat right and into so like this kind of remnant of our evolution um it’s kind of not that you have to kind of fight it a bit uh then your brain is is a bit better I think so if you’re an adult here like you can do it it’s never too late I want to switch gear into um Um, maybe it’s a bit too niche of a topic because I really like learning. Like it’s like something Go for it. I’m happy to talk about anything. I really like and um uh I I’ve seen you and then like uh the rest of the the Met Academy folks talk about mastery learning a lot and um I was reading your your 30,000 hour like the self-directed intense uh type stuff and the first thought that come into my mind was um uh ultraarning. It’s like by this dude Scott Young, right? Um, and yeah, like he he branded the whole thing, but like at the core of it is that it’s kind of an intense way of learning to do like a specific specific skill. Um, and he went through the same thing as you. Like he he did that MIT courseware and he was trying to blitz it all in a year and all of this thing. And then uh there was some stuff that like he was able to kind of like uh do all of the exercise, all the exam and all all this stuff. And uh when he was actually doing the rigorous type of of grading, he got a bit lower um result than he had before when he was self-grading, but it was still pretty good. So I think there’s a lot of uh of of connection. Well, first of all, are you familiar with like this whole ultralearning stuff or or not at all? I I’m a little bit familiar with it. I I have the book on my my reading list, ultraring book. I I’ve I’ve um I’ve read through like a few blogs on like the, you know, his his his journey uh through this. This was a while ago, so I’m a little a little rusty on on all the details of it, but I I mean this this comes up very very frequently when when I when I when I talk to people, they’re like, “Oh, this is this is very similar to the Ultralarning thing.” And I’m like, “Ah, yeah, that’s that’s right. I I need to read this thing again. I need to like retain it.” It’s funny because it started the same way, exactly the same way as you. So like uh uh like but uh uh you went the other way at the end like you went through like mastery learning which is a bit different than like what what this thing is about. um which is like you have structured like efficient like uh no no real autonomy about the sequence about where where you’re going and then you build this as fast as possible. But like when you were doing the the 3,000 hours um it was kind of a bit the opposite where like you had this goal and then you were kind of doing like the course in a more exploratory fashion. Um how do you recon these two? actually. So, when I was when I was uh doing my own version of the the ultrallearning marathon, the 30,000 hour study marathon, so I was um I was actually not really going in my own interest areas. This was I I think this was a little this is a difference between me and uh my my program, the ultrallearning program, how we kind of started out. So, my my and I also did not have a very concrete goal at the end of it. So my my goal was actually not to go through the entire uh or it was not like a specific project that I wanted to do. It was not it was just like I I I this started with me realizing that like just an earthshattering realization that the pace of school is comically slow and if I just take this into my own hands then then I can learn way faster. that paired with like just the idea of like okay all this um like so initially it was like it started out I was like okay I’m going to learn calculus that was like the the the initial goal um on but once I got through that then it was like I’m just going to like all this math stuff is it seems like it’s it’s going to be really useful for for me in the future and I’m enjoying it like this is a lot of fun like I just and improving at the stuff it’s a so I I didn’t have a a concrete area that I was just like, okay, I just want to learn like this this subject as as as well as I can as as quickly as I can. And so it’s not like I was like sitting there like being like, okay, um I learned calculus, I learned uh some linear algebra. Why don’t we go into differential equations? How about that differential equation? How about that one? Oh, that’s cool. It’s like what about that cross-pollination between differential equations and and some uh like in a multivariate case a partial different like I I wasn’t like um like going on Wikipedia and going down the the rabbit hole on this stuff and and backfilling. It was it was a very structured kind of like I’m going to like I I I just just tell me what to do. Just find a resource that just tells me what to do in the most efficient way possible and I want to do it. And so initially like you know this I mean I I was coming into it kind of like I I I I wanted to be efficient but I wasn’t always going about that the the right way. Like I was hitting my head on like I my my end goal was like I wanted to learn this stuff really really like I want to get as far as I can. You you you were going for the mastery of the skill, right? Yeah. For the sake of the mastery of the skill. uh which I think I think you’re right that this is the core difference because like um in the ultraring kind of paradigm it’s kind of the uh um the opposite in a some sense where you always have to have a goal if you have no goal like it’s not going to work like you’re you’re there’s nowhere to anchor you’re just going to go aimlessly around right so there’s a goal and then there’s like this kind of prerequisite chaining kind of movement where you try to do this stuff it doesn’t work you go back like what’s the prerequisite for this and then this doesn’t work you go back and then then you go back and you go back. So you’re right if like um with the discussion that we had before um uh like this there there is some issue with the the top down approach um if the goal is not well defined and if the gap is way too big like if you have you never seen like like admission and subtraction and you’re trying to do the the top down learning um is at the end of the day it’s going to be a waste of time because there’s just so much prerequisite that you have to uh uh to go through um and I think that’s where like there there’s like the the disconnect um uh between like what you you were uh you were doing and like what the something like Ultra Learning and Scott Young was doing like they had a specific goal in mind and they were trying to blaze through it as fast as possible. You know, one thing I I I I want to say is that like sometimes um so I’m never if anyone has a a particular goal that they want to achieve like in machine learning I want to reimplement this paper or whatever like I never want to say like don’t even try it like you need like go fill in your foundations like like I’ go try it give it a whirl like if you are close enough like within striking distance that you can do it then like who am I to like say that you you shouldn’t I’m I’m just saying like if if somebody is like man I tried it, but I can’t do this this this thing. It seems I What? What? What the hell do I do? Like, am I just too dumb to do it? Then that’s when I’m like, “Okay, no. Stop what you’re doing. Get your foundations in place. This is going to make all the difference in in in the world.” Yeah. And I just like get the foundation like really slowly, you guess, as efficiently as possible. So, like I I don’t even think like there’s um so much disconnect between like a mastery learning and ultraarning. Like in in in mastery, you’re trying to go to for the mastery of the skill. So you have to make sure that you have the right foundation at the right place and you’re honing that stuff, but like maybe you’re already mastering enough to do the thing. So you can just do it and then like you continue and then build up the mastery. Um and like this whole question set like got sparked because of that biology stuff that you were doing, right? um it really was looking like um some ultraarning blitz that you were doing and then you were doing the prerequisite chaining and like every time the LM was like hey what do you want to like you’re like dude I’m focused on this and we’re going to that goal. In my view this was more on the on the on the edge of a of ultraring kind of sprint project uh than the mastery type of uh skill set building. Yeah, that’s interesting. So I when I was doing the biology stuff, I I had kind of viewed it as like the math academy approach to just what do you do if there’s like literally no infrastructure. And so I I kind of viewed it as like, okay, I just want to spin up on biology as as fast as possible, as efficiently as just master the subject. But but yeah, now that you bring it up, it’s it’s true. I kind of scoped it down in particular to, you know, uh, genetics cuz that was the that was the the area and and um and so I I think overall like there there there is a reasonable amount of scoping down that you can often do through top down planning whenever you you have a goal. And so that’s that’s something that we’ve actually um we’ve created courses on on math academy specifically for that purpose like math for machine learning math for mathematical methods for physics where it’s like you know if somebody wants to do machine learning you know you you’re going to have to learn linear algebra multivariable calculus probability and statistics but you don’t need to learn all of each of those courses like there’s a lot of multivariable calculus like Green’s theorem divergence theorem Stokes theorem like at that come like towards the end that’s like you know that’s not the type of multivariable calculus you need like really just need up to like the the chain rule and and and like hyperplanes and and stuff like that which is really I mean it’s a it’s a large chunk of multivariable calculus but it’s it’s not the whole thing and so you can there’s a lot of reasonable scoping down that you can do with that that’s kind of like a almost like a hy I guess it sounds almost like a a midway between like just the the entire subject mastery learning versus the only prerequisite chaining um ultraarning to like just a kind of like a a scope it down top down and then fill it up bottom up sort of sort of thing. And like to be very honest like the ultrarning like uh type of learning is is very efficient in order to get to your stuff but like your foundation is still brittle, right? Like there there’s no escaping you not become master in math, right? right? Or like master of like language because now you can speak it. Um you just are able to do it, right? Um which is very very different uh than like a fullblown mastery. It’s more efficient to get the goal and it’s faster. It’s more rewarding in my view also if you’re going at it at the right way and then you understand like you actually have to learn stuff like you’re not going to just wing it and like it be done. Um yeah. So um cool. Perfect. So I got my information here. The last topic I wanted to bring in is um kind of more on the like u at the French right at the at the the not typical type of stuff right I I have a question specifically for like the neurode divergent uh people because there’s a lot of them I’m not neurode divergent like I actually uh um I don’t have any problem like um focusing for very very extended period of time um and it was always a trouble when I was like uh speaking to people with ADHD and that they were with me and stuff because what I was doing was not fitting at all like what they could do and not just in like yeah just try better. It was like almost physically impossible to do the same amount of like I could stay for like 6 hours like not moving and just like turning like a whatever it take in term of like math problem right it didn’t matter was reading and stuff the only the my only issue is I had to eat and at some point I was getting tired so I knew like my brain was all [ __ ] up but I need to sleep for 10 hours right that was the stuff that I could do right I could do that for a week no issue right and I was uh showing that to like my brother and like my my my the other people in in my in my extended family that like um were more in the idiot side. It was kind of impossible for them to even conceptualize like this is this is possible like how how would like these type of people that have more of a ADHD ADHD type of of neural profile uh would fit in your in your model? Yeah. So, you know, the the interesting thing is um a lot of these people with with with ADHD um think that the you know, just a lot of the the education system is is not cut out for them. Um and and that they have like a you know that they’re just not geared for for optimal learning and and stuff. And so only one of those things is true. The education system as it stands is not friendly to to to people with uh with attention challenges. like people who can’t sit still like actually I mean it’s it’s really not friendly to to to anyone but especially people with attention challenges and and it’s like nobody likes being talked at for an hour on a subject but a lot of like I’m kind of similar to you in that like I can just like you know I I can grind on something for like six hours before I even realize that any time has passed and and um it but it it’s like yeah but it if you take uh somebody who who has like just you know more is is not like you you and me in this opposite way. It it it like just the impact of being talked at for an hour or like it just kind of limits the the amount of um you know how how much can you communicate to somebody before they start doing something. And you know, initially that might seem like uh some some kind of disadvantage like well oh I I I just can’t sit still for like for the hour it takes to to go through a lecture or whatever like you know but really like if you look into like the you start going down the rabbit hole in the science of learning it’s like well that’s not even a good thing to do in the first place to be talked at for an hour. That’s like it’s just people have a some other people have a higher tolerance for it but it’s not good for anybody. What’s what’s good is these minimum effective doses of just like okay just tell me a little bit of what to do and then let me start doing it and then like you know just active active active the more you stay just like in a sort of passive state or just like not doing anything it’s like you’re not you know if if you go to tennis lessons and the coach just talks that for you an hour like I don’t care if you have ADHD or not like you’re not getting better at tennis like if you have ADHD then you might like just be expressing more like discomfort like come on let’s go let’s let’s hit some balls do something right but but either way the learning is not happening so I think a lot of times like the the neurode divergence stuff just makes you more sensitive to like things that are like there’s an issue here it’s like you you feel this the the the issue more more more earlier on but um so you know I mentioned I I am very similar to you in in in that I I I can uh you know the passage of time is is is just it’s hard to to to keep on on track of um I can just like sink into somebody. Uh our our founder of Math Academy, Jason Roberts, he he is actually on the and in the exact opposite end of the spectrum. We’re in the ADHD spectrum. So, a lot of mathmies is is just it’s built um in terms of what would Jason have want what did Jason hate about math education growing up and and like you know and that was one of the things like being talked at for a long time before actually doing problems like just hated it because like you know he could he’s got these stories about like he was just be sitting in class he could never sit still the teacher’s like Jason pay attention like what are you doing and it’s like well the problem isn’t Jason not paying attention the problem is that like he’s being talked at for too long without doing anything. And yeah. Yeah. And so Jason actually has uh three kids who are all across the different spectrum. One uh was um just like very uh highly computationally mathematically uh gifted and just like blows through all this stuff. Um, and he has another uh kid with discalculia and then another kid who’s who’s kind of just more of your your typical um honors math student level. And so the this the the kind of approach that we take is is kind of, you know, making sure that that this efficient learning cycle is working for for all the the students. And you know, he and I are he and I are so different um too and we just agree on like all the same things. Like it’s just whether you come at it from like okay kid with ADHD can’t doesn’t want to sit still for like 10 for for 15 minutes for an explanation or you come at it from a just a calculate the learning efficiency of this setting versus that setting you get to the same answer. I think I think like the the the optimal learning uh flow I think it’s kind of the same in in all I think the issue with most um uh AG folks is that like at the at the ground level they have like this dopam dopamineergic dysregulation but there’s also like all of the other emotional component that they’re layered on top. I guess this is a very it’s not like just like hey I have a problem like your brain is is not the same as like neurotypical like it’s not thinking the same way right and it’s fine to have like all of the different profile but what I see often is that like they they have this shame of like not being good or like not being able to sit there and then like listen to it. So then they develop all of these kind of uh like bad emotions with the thing and like it’s not it’s not about the thing anymore, right? It’s about how the thing make them feel. And this is is an issue because also like um if you like uh look at these people, it’s not they can’t focus, right? It’s their focus is not like as evenly spread um as a neurotypical. Like in my case, I could be like learning about anything and it’s going to be fine. like uh we’re going to like learn about ballet. I’m going to listen and we’re going to try to figure it out, right? We’re going to learn about like rocks. We’re going to listen. I’m going to figure this out, right? But like in in in in most of these cases, if it doesn’t fit into like the cone of interest um that they have, um it’s going to be very difficult. But when it fits in there or you make it fit for some reason, right? Making it easier to ingest into that little comb. Yeah. Um like some of them could could sit still for like 8 hours, no issue, right? Like I’ i’ve seen people that like their thing was coding, right? They like they they this is the stuff they really liked front end web development, right? For some reason, this is like something that they were highly worried in. You talked to them about operating system or other topic like they they kind of zone out like, “Yeah, but like it’s not visual. I want to click it on the stuff and like see the things. Um they could sit there like there’s no issue about it. It’s just like not as evenly spread out. Um so if you run into like these these like talking session and they can’t see and how to kind of ingest into their stuff what doesn’t fit in there and they they pile up all these emotions. Um then it’s you have to deconstruct that before doing anything else. Like I had student like this like they like it was a like a psychiatry session for like the first three four session like no it’s not you it’s like this and like you see and then like after some time like then we pick up pace and then we were able to do it in a one-on-one session but if they didn’t have that or like they didn’t have like somebody with patience um they just they just will feel dumb and that’s the end of it. Absolutely. Yeah. It’s it’s kind of funny the the feeling of it’s like the the feeling of dumb is often the real impediment to the learning. Not not the not the being dumb. Like often usually the the issue is not that you’re dumb, it’s just that you like some other settings that are are not, you know, causing the information to flow. Whether that’s a prerequisite gap or just a like, you know, just not wanting to participate in the in the process or or or whether it’s you’re just feeling that like this is not you. You don’t identify with with this stuff. This is not something you’re good at. It’s all these kind of blockades going to be put up and it’s just this is separate from the actual like how good you are at at that. I think that’s actually one of my superpower like um I’m okay being like looking like a dumbass because I know that like that’s just normal like it’s you’re you’re new to the field right you’re you have to learn all this stuff yeah just just be dumb fast right and then you’re going to be good and it’s not going to be okay like the guy will laugh at you because your question is like absolutely basic okay now I know it and like I don’t care about this guy I’m here for this stuff right so let’s learn it and if we switch gear to like like how uh the new technologies ies are like entering a bit like um the picture now. Um do you think that if let’s say there’s a gifted amateur mathematician and like he’s doing academy and stuff and then he he has all these like emerging tooling and and like could use AI to kind of probe a bit on the frontier. Um do you do you think we can see like some more like amateurish mathematician be able to contribute to the frontier? um or still like we will need to have like uh this kind of formal uh mastery learning process. Yeah, that’s a it’s a really good question. Um all right, so I’m I’m not a research mathematician. So there is a I I I don’t want to like overstep in in in my kind of answer to to this making claims that I I don’t actually know what I’m talking about. So I I think I think that this this question would be would be best posed to somebody who has like you know a track record of of mathematical publications. Um, let let me let me generalize the the the scope here a little bit about just like amateurs like contributing to the edge of fields and in general. One thing at least in my experience um and that I hear from a lot of other people confirmed is is is that this tooling that exists nowaday like LLM uh stuff uh cloud openai whatever um all these kinds of tools are they’re kind of multipliers on on what you’re able to to do initially and so now like if you’re not able to do a whole bunch uh you don’t have like technical chops and and And then you you you you ask like some stuff ask questions to the LMS have it like code up stuff for you that you couldn’t otherwise do. This can feel like sort of an like an equalizing like oh hey like I’m at the same caliber as like everybody like hey I I don’t have to learn how to how to code very well because I can just vibe code everything and like I don’t have to learn pedagogical like like cognitive science and stuff. if I can just tell like tell tell tell the LLM to to to generate like a a a course for me or like stuff like that. It’s like well okay that it is I guess leveling you up but now like think about how much it’s leveling up people who actually know like these foundations of what’s going on and like the thing is like when you have uh a high level of mastery in something at least in my experience um these tools can scale you like you you are you are you are the zero to to one and then these tools put the the zeros behind the one go from 10 to 100 000 and if you don’t really know what you’re doing, you know, you can’t it’s like you you you start being led by the LLM exactly into it will lead you in weird direction like they like it will lead you like cheerfully into like some psychosis area where you’re like what the [ __ ] is happening here. I receive a lot of these DM like hey look I made this and like 26 pages of like what the hell are you are you even saying right? Like is it good and like Did can did you try? Did you run this stuff and you’re like no but like am I on to something and but like and then you see like on on the opposite side like um when you you do have a skill if you look at somebody like Terren to he’s actively using these tools and getting a lot of out of them because he he’s really good at this stuff like is one of the best right and he’s able to kind of leverage them to kind of like like help him scaffold and like move a bit faster. Um, but still like it’s his intellect and it’s like his mastery of the material that is able to like figure out when they’re going wrong or like like the the question is not well posed or like this the there is something missing in what he’s saying or like there’s a gap or something they didn’t know that I need to bring into the picture and okay yeah for true I remember that stuff is is for this and then you can you can kind of pile pile this stuff together but um I agree like on their own like uh uh it’s very difficult for for a beginner to not get like absolutely swayed out, right? Like they will not know where to go and like what’s what’s truly novel, what’s is what is at the edge. Yeah. Yeah. Absolutely. Yeah. Yeah, you know, a lot of the the efficiency gains that I’ve seen personally using these these tools have been it’s like I mean I’m not it’s like it’s more it’s more about delegating work that I already know how to do that that I just is that that I can automate more and and and just free me up to do additional stuff. But it’s like it’s this is not a a way to get me to to to to offload work that I don’t know how to do because it has the same failure modes of like I I’ve done this before. I made this mistake before of like trying to like I have not have a you have a junior employee, human employee, and you got a project that you’re like, “Oh, this sounds pretty cool.” You haven’t really scoped it out. You haven’t like gotten in the weeds on it or anything. You’re just like, “Oh, yeah, this sounds like like a good thing for you to do.” And then you like you you you look up a day later, a week later, and you’re just like, “Wait, what? Hold on. What what the hell is happening? Like this is all my god.” And and then you have anyway. Yeah. you didn’t prompt it. Well, but it’s true. Like they go into the weird direction and if they’re not asking for feedback, um they’re going to go into like some weird stuff and then you’re going to unblock them into like two or three sentence and they’re like, “Oh.” And then you’re like, “Yeah, man.” Like, and if you can’t give the feedback, if you don’t know, Yeah. If you don’t know the feedback, you’re like, “Yeah, just go and do it.” And then they’re going to try more and they’re just going to burn out like this is not help. Yeah. Yeah. I I also use these tools um like when I know it’s it’s good but also when I don’t know I it’s useful to get like a lay of the land like 80% correct right and like yes they have their own tone and their own assertive seriveness. If I strip that all the way I’m like okay there’s like these stuff perfect and then you go and you drill into the right things and you learn them properly but then it quickly give you like a good uh approximation of like what this stuff is actually looking like. Um yeah. Yeah. It gives you like a little lay, okay, here’s our options here. What do you think? And then you can kind of like evaluate all those. And totally totally agree. Yeah. You know, one of the the big unlocks I’ I’d say that I’ve experienced recently at least with the um you know, agentic coding stuff was was kind of just um scoping these scoping stuff down enough that I can that I can get it into like that it fits into my own working memory of of what we’re doing here. That that was one of the challenges that I experienced before is like like I have this whole plan like um for like okay here’s here’s the the feature that we’re going to build here’s the thing that we’re going to do. You ask it to do too much and then it’s going to go off in directions that you don’t anticipate because you have not like verified everything that it’s going to be doing cuz you can’t keep it in your working memory. What if you like split this up and to plan like if you are able to like sign off give expert feedback and everything. Okay. Yeah, this all looks good. I’ve seriously looked at every little thing and it looks good and you could keep the the rains tight like that then yeah it’s amazing what these can do and it helps like like I I spend a lot of time like uh working with the thing to get the right blocks like mapped out right and u I usually like make it write report and stuff and then I look the report I trimmed out I removed like [ __ ] it says and like simplify they wanted to go into big direction like if you do that like it’s going to cost us like thousands of dollar from nothing just do this thing and it’s going to be good and I iterate like this and I I iterate with it to kind of re revise the plan and then start to chunk it out but then I take a fresh session I’m like we only going to do this stuff right this is the only thing we’re going to do and then it does it and then you review it you change a bit of thing and you’re like got done right then you take the other things but like all of the architecting and like the the connection to like what you’re actually trying to do is literally you done like uh you done all the duct tape to put to pull everything together. That’s much more efficient than uh than just trying to to let something go when you have absolutely no clue. You’re going to get something out, but like like it’s it’s very brittle in uh in in structure. Perfect. Man, I had another question, but uh I think I think I think we covered this in like six different ways already. Um I wanted to ask you like u to close like what are you most excited about at the moment? like in term of like learning meta academy math in general like what what’s the thing that excites you the most in this in this area? Yeah, you know there’s there’s one thing that I I think about as soon as I wake up late into the night all the time it’s what I’m working on and I I I think I mentioned at the the very beginning when when you ask like what am I working on right now? It’s taking Math Academy from workshop to a factory transition. It’s a huge team lift with me and Jason and uh Alex, our director of curriculum and like just figuring out how to you know take the it’s it’s like it interfaces with learning with like um all the AI tooling everything how to how to deliver this this this learning experience and in particular this this this content uh the courses at at a at a much higher level of scale how to turn out more courses like a like a factory style, but not like, you know, not not just like, hey, I’ll build a course for you and then it’s slop. Like, how do you how do you take the expert take the human expertise and and apply these tools as multipliers to just speed up that level of expertise without dropping standards? And we’ve made so much progress on this in the uh in the past few months. And I’m I’m really excited about this. But this is really our kind of like, you know, the Tesla Elon sleeping on the factory floor kind of stuff. Just like the production hell of like, oh my god, this is we got ourselves into this game and and now and now we have to like scale up and really deliver and just it’s going to suck for the next however long it takes to make this factory work. That’s kind of where we’re we’re kind of doing the the equivalent. That’s that’s where we are in our journey right now. So that’s just that’s that’s what I’m doing like every waking hour basically. Cool, man. Best of luck. Um and um honestly I think like there’s there’s nothing more satisfying than to build something like truly at the edge like this not done before and then you’re trying your best and then you’re using all of your skill set in order to get there and and do the thing. Um that’s I I always felt like this was the most exciting and this is also where I learned the most like I learned the most about like how to do it faster, do it better like uh u and then just ask for the right help at the right time also. Um yeah, fantastic. Thank you assistant for Thank you for having me two days to be on. This is great. This is a lot of fun talking to you and you know like I said earlier it’s only I came in with reflections today and that only happens when you have a really good conversation with somebody. it it’s so the testament to you as a interviewer here for putting so much effort into into this coming up with great questions and uh well also answer including so many people’s questions that they sent to you like I I I I never imagined that that we’d be addressing this many people’s questions and it’s awesome that we were all the possible question like like this will be well packed like we can reuse this stuff for like the next 10 10 years I think we’re going to be covered And that’s it for today, guys. I hope that you enjoyed the video. And don’t forget to like and subscribe if you enjoyed this stuff. And I wish you all a great