I haven’t written in a couple of months as we had another baby (kid #3, a boy) in January. So far he really likes it when we vacuum or play Metallica… not sure what that says about his personality but at least the house is clean. Also I’m still working on a major update to teachyourselfcs.com so please keep your helpful suggestions coming!

A number of my students rave about Math Academy, an online math program that claims extraordinary outcomes. It’s rare to find a discussion of math education on Twitter or Hacker News now without somebody chiming in to recommend it.

The reviews from thoughtful math teachers, though, are not so positive. Michael Pershan feels that it is “fundamentally broken” and can’t be fixed. Dan Meyer suggests that the learning may be illusory but that It Is Fun to Pretend That Hard Things Are Easy! Math Academy employee Justin Skycak was the most requested guest we’ve ever had for the CS Primer podcast; after his appearance, the private response from some of the educators I most respect suggested that I should have given him a harder time!

Education is complex, so it’s common for the most beloved courses and books to also be the most criticized. Structure and Interpretation of Computer Programs springs to mind, or the Dragon book. My goal here is not to resolve the conflict so much as to tease it apart, hopefully in a way that’s interesting irrespective of whether you’re in the market for a drilling app. Admittedly this will take some time (3000 words, 15 min reading time) so:

TL;DR:

Math Academy can be an effective and fun—even mildly addictive—way to drill procedural fluency. If your goal is anything more than test prep through, you should probably find a textbook or lecture series that moves you, for depth of conceptual understanding, and only use something like Math Academy as a supplement. If the Math Academy team were to just acknowledge this limitation and perhaps suggest good accompanying material, I would find it easier to recommend.

Any fool can know. The point is to understand.

— Fictional Einstein

Also a caveat: I’m concerned that by giving anything other than a rave review of Math Academy, I’ll discourage those who are finding success with it from continuing. Please, if you like Math Academy already, don’t quit!

A possibly addictive feedback loop

The core structure of a Math Academy lesson is a worked example, followed by a few typically multiple choice questions. Explanations are minimal and motivating context virtually nonexistent. The experience is roughly as follows:

MA: Let’s say you wish to differentiate an inverse reciprocal hyperbolic function

You: Ok…

MA: The derivative of arcsch(x) is […]

You: Ok I’m sure that will come in useful later

MA: Yes it’s an important node in the dependency graph of all of mathematics, you need to master it to progress. Now, here’s an example of applying the formula for a trivially different function like d/dx arcsch(3x)

You: Sure, that makes sense

MA: Follow those same steps, for these further slight variations

You: Ok…

MA: You get 10 XP! Keep going to maintain your streak! Just 100 more XP to get promoted to the next league!

The funny thing is that it does actually feel good to earn made up points and move up a leaderboard. At one point, I was racking up enough XP that a stranger on the internet noticed:

Having someone ask if I was “hooked” frankly rankled. Yes, I was enjoying the process, enough that I kept going beyond my original goal of giving the system a good try. But did I want to be hooked on this? It was fun, and the XP showed that I was (re)building procedural fluency in a topic that I do think is worthwhile. Certainly, it would be better than being hooked on some other kinds of fake internet points. But there is more to mathematics than being able to follow a sequence of rules, and I was concerned that my attention was being captured in a way that I did not in fact want.

Look where procedural fluency got me

The reason I could startle strangers on the internet with my “insane numbers” is that I’d been through this process already, 20 years ago. In high school, I studied hard for my final exams including for an advanced course which covered much of the same material as the early undergraduate level courses on Math Academy. I did well enough that I was accepted into a highly competitive program at university, intended for the most promising math students… which I promptly discovered was not me.

See, I had “earned” my place by practicing enough to score well on tests without really understanding what I was doing. Another student in the program—actually, the one who scored highest in the entrance exams—was in a similar position, and mentioned he had worked multiple times through decades of past papers, in preparation. He ended up becoming an engineer, and I ended up doing whatever it is that I do.

Meanwhile, the two people from our cohort who did become professional mathematicians were entirely different: they didn’t drill at all, but instead interrogated gaps in their understanding, found novel ways to develop their intuition, and overall focused on subtle conceptual aspects of a topic that could never conceivably be formulated as exam questions. They still did well enough on exams, but that was an afterthought.

The gap between us was confronting, and apparently insurmountable. I quietly gave up, and mostly coasted through the rest of my degree, taking as many computer science topics as I could through the math department and hoping that I would learn the lesson for next time.

Shortcomings of the diagnostic tests

Since my time at school I’ve come back to a few math topics with more of an “understand what I’m actually doing” approach, but I still have a lot of gaping wide holes, including embarrassingly in calculus. This came up recently as a barrier to my current study of biology, as I wanted to understand certain processes that are well modelled as differential equations. So in trying Math Academy, I had in mind that I might also do a better job with calculus this time.

I wasn’t surprised not to be able to test out of Calculus 1 on Math Academy, although it was telling how poorly the diagnostic captured the core issue:

• Many of the questions I answered correctly were in areas that I don’t feel I understand well, but where I suppose I had drilled enough in the past for my procedural fluency to survive 20 years of atrophy.

• Each question is timed individually, so some of my correct answers were too slow to count. These tended to be in areas where I do have some fundamental understanding, since I was able to re-derive a relationship I had forgotten, or reason about a problem creatively, albeit slowly.

• Some of the questions I got wrong were those where I felt like I understood the key concept well enough, but misunderstood something about how the question was posed, or made two thoughtless mistakes (if you make just one, they mercifully allow you to try a second problem). One slip up here cursed me with a long sequence of middle school trigonometry problems that were excruciatingly easy but couldn’t be skipped, to the point where I just retook the entire diagnostic. I will say more about the hubris of the “way” in a moment.

All of this is to say that the concept of a diagnostic test is great, and it does a fairly good—if frustrating—job of testing procedural fluency. Understandably, it does a poor job of testing other things like the depth of your conceptual understanding, or your ability to generalize. This should serve as a reminder of what you should or should not expect to get out of the platform overall.

The hubris of the “way”

Speaking of highly hyped educational products with major shortcomings, my 6yo daughter occasionally uses a program called Synthesis, and quite likes it. It claims and of course fails to be a “superhuman math tutor” but thankfully it’s just a collection of thoughtful interactive explainers and mini-games. The most common complaint I’ve heard about it is that there is not yet enough content, so they are doing ok!

Lessons are presented as conversations “with” the Synthesis “tutor”, which in practice is a few sentences of hardcoded prose followed by a multiple choice question. There is clearly a main path through the lesson, so most “choices” lead to effectively the same place at the same pace. This is not altogether terrible: I’m glad they thoughtfully designed a main lesson sequence, and the interaction mode can be cute and engaging.

That said, it is frustratingly the least “adaptive” of any math app we’ve tried. It is the only app where long sections can be both mandatory and so mind numbingly obvious that I need to speedrun it myself to keep my kid’s interest. In theory, the “interactive” structure should be more responsive than a traditional program; certainly the human teachers that some Synthesis ads condemn would allow a bored kid to progress faster.

All it needs is a “skip lesson” button! Why is there no “skip lesson” button!?

My suspicion, having made similar mistakes in the past, is that they were too confident in the brilliance of their solution to recognize its shortcomings. A “skip lesson” button can feel like an admission of failure, and hey maybe the brilliant original idea can still be salvaged.

Math Academy fails with similar hubris, although to their credit, they do seem to be making an earnest attempt to smooth the rough edges. Allowing a second attempt at a failed diagnostic question is a good idea. But subjecting someone to literal years of unnecessary remedial “pre-requisites” is too great a punishment for a second slip up. So too is requiring review questions on a topic after any incorrect answers in a quiz: a human tutor would focus on conceptually significant misunderstandings, rather than trivial errors and typos. The justification from Math Academy for this design is likely that the program is based on “mastery”, but the kind of mastery I care about is about developing my intuition, not honing my mechanical precision.

The DAG is a lie

One of the biggest falsehoods in education is that everything we might want to learn has its tidy little place on a dependency graph. Want to be a machine learning engineer? Of course you should understand how the algorithms work, so first learn linear algebra and calculus, for which you need algebra, so of course pre-algebra, and so on. Don’t forget to do it on the systems side too! Perhaps start with CUDA, no wait computer architecture, no electrical engineering, no physics…

Look I understand the appeal. As a teacher it’s nice to be able to establish some assumed knowledge for a course so that we can build on from there. When I teach operating systems I like to assume some understanding of computer architecture so that I can explain a context switch, say, at the level of the CPU. Similarly I wouldn’t encourage you to teach calculus before arithmetic.

But should you learn integration before differentiation? I’d never considered this approach until I read David M. Bressoud’s fantastic book Calculus Reordered, which argues among other things that the conceptual basis of integration—as accumulation of thin slices—is a more intuitive starting point than tangents to a curve or rates of change. This was certainly the case for Archimedes, who computed volumes this way two millennia before Fermat and Descartes started finding tangents.

Talk to high school math teachers, and you’ll see that it’s also problematic to start with integration: problems can quickly grow too challenging algebraically, deviation from the standard sequence can leave you without much external support, and students can lose motivation when the applications move beyond areas and volumes.

Of course it’s not just differentiation vs integration. For any customary “dependency” of one topic on another, it’s quite reasonable to consider the inverse:

• If I taught operating systems before computer architecture, would that better motivate the latter?

• If someone’s excited to learn about deep learning, should we really tell them to go away and do 5 years of math first? Jeremy Howard vehemently opposes the standard bottom-up approach; his fast.ai course is extremely successful and top down.

• If my kids learnt physics intuitively at a young age, for instance in the style of Lewis Carroll Epstein’s Thinking Physics, wouldn’t this be great motivation for calculus, later?

• If data structures and algorithms were hypothetically the final course in an undergraduate computer science program rather than one of the first, wouldn’t students approach it with much more contextual understanding? And would it really be so bad for the others?

Ultimately there’s no one best way. Ideally each student would bounce between topics based on their specific background and goals, a level of flexibility that’s challenging for a school but easier in a less structured environment. If the learner is unconstrained by structure and driven to succeed, they’re served best by the meta skill of knowing when to switch focus from A to B, and when to return to A. Chess grandmasters don’t “master” openings before moving on to middle game and then endgame; all aspects of the game are studied continuously, and each informs the others.

So why do some learning resources designed for autodidacts—such as Math Academy, or the generally very good Execute Program—rely so heavily on dependency graphs? The generous answer may be that it’s unrealistic for the learner to know an appropriate ordering, and perhaps motivating for them to be shown the “ideal”. The cynical answer is that these programs take the idea of mastery learning too far, and have become dogmatic. The hubris of the DAG. After all, it’d be easy enough to present users with a suggested sequencing without strictly requiring that it be followed.

This is all part of a broader gripe I have with “mastery learning”. At first glance it all sounds well and good: yes surely we should “provide students with individualized support and repeated opportunities to demonstrate mastery” before moving on. But what happens if the student is still not passing, say due to poor curriculum design, or insignificant issues like test-taking skills, or simply boredom? Do you tell them to just try harder or give up?

I recently read Jonathan Watts’ eye opening biography of James Lovelock, a world class scientist who among other things invented the electron capture detector and first observed increased levels of CFCs in the atmosphere. Lovelock nonetheless struggled with basic arithmetic his entire life, requiring some impressive workarounds:

How would “mastery learning” have treated James Lovelock? The school system was bad enough already; thankfully he had the confidence to endure its poor judgment of him for long enough to prove himself in other ways. But if you were to place elementary school arithmetic on a dependency graph of all of mathematics, it is surely a very early vertex. If someone like Lovelock simply could not progress from there, it is a failure of the learning system, not the individual.

Social movements are good, actually

Despite these structural issues with Math Academy, it is still a great option for many learners, as evidenced by hundreds of happy users and frequent rave reviews. I wish Pershan and the other detractors had at least acknowledged this: all these students are now doing math on a regular basis, when they simply weren’t previously. What reason could we possibly have for discouraging them?

Sure, much of the positive sentiment is due to buzz. Somebody may read a Twitter thread by a Math Academy employee and test it out. They enjoy the feedback loop for a while and recommend it to others. Before long, people are recommending the program without even trying it, based on a halo of novelty and general social interest. A textbook may be much better than the corresponding Math Academy course but unlikely to generate the same level of hype.

But this is fine, even good. We are social animals, often with social motivations. It can be fun to participate in something that others are excited about, and buzz can help provide the activation energy for trying something new. Some people will stick with it for a long time, others cancel after a month, but this is simply how things are with educational products: everyone arrives with different context so no single resource will be universally ideal. If social buzz encourages a few more people to try it, great.

Understanding the glories of civilization

I’m not very socially motivated, myself. I’m glad that the hive mind suggested I try Math Academy, but social buzz and leaderboards won’t sustain my interest for long. Nor are points, streaks or other game mechanics. These may provide a little activation energy, but if I’m going to invest hundreds of hours into learning something hard, and retain what I’ve learnt a decade later, the motivation must come from elsewhere.

Every person has their own mix of motivation and the last thing I want to do is to judge another’s. But speaking personally, if a book promises to help me grapple mathematically with the chief glories of Western civilization, now THAT might sustain my interest.

This is from the introduction to Differential Equations with Applications and Historical Notes by George F. Simmons, a book as close as possible to perfect, for me, for this topic. It is not just motivating but stirring, driving, elevating. By math textbook standards, it overflows with fascination, excitement, even love.

The Simmons treatment is so vibrant that something like Math Academy sits lifeless in comparison. It feels rude to use them side by side. But that’s likely what I’ll do.

My fundamental gripe with Math Academy is that it overstates the value of procedural fluency. I wish they would conceded this, and at least suggest conceptually rich Simmons-tier books as supplements. Personally, I see the textbook or lecture series as core and Math Academy as the supplement, for extra drilling where needed, like a digital Schaum’s Outline. Resources like these are highly worthwhile but ultimately limited, so if you’re taking responsibility for the outcomes of your study effort, you should remember to mix and match as needed.

… anyway, it’s been a while since the last of these newsletters! Instead of trying to sound profound enough to make up for the silence, I’m going to try a new format that should be easier both for me to write and for you to skim.

Thoughts on AI coding assistants

I generally prefer not to comment on software development practices, because of something I’ve observed often enough that it feels like a law: for every excellent engineer who swears by a particular practice, there’s an even better one who swears by the opposite. Some people couldn’t imagine coding without unit tests, or code review, or continuous integration, or step-through debugging, or [your preferred “best practice”]. Yet, there are people out there who do the exact opposite and outperform us all.

But people keep asking for my take on AI coding assistants, and with the caveat, again, that there are better programmers than me at both ends of this spectrum, here is my current practice.

For almost all meaningful tasks, I don’t use AI. I use my brain, pen and paper, a minimally configured text editor, and something to run my code. I don’t like something like Cursor for most tasks as it feels like a distraction, as if I’m pair programming with an intern with great recall but poor judgment. To compensate for my imperfect memory, I mostly stick to my ingrained habits of reading official docs, man pages and source code, or just running quick experiments.

I have enjoyed using Cursor—or more often just Claude via the website, since I don’t usually want a whole new editor—for smaller projects, like translating short programs into an unfamiliar language, writing throwaway personal tools, or learning a new library or domain, where I can ask a lot of questions and run faster experiments. I know that the “poor judgment” aspect of the LLM is still problematic here, and that I may have to update my mental models as I go, but I see that as an unavoidable part of learning anyway. I would far prefer an expert human pairing partner for such things, but they too will teach me imperfectly, and in any case there’s not always one around.

I’m also aware that I have some baggage, or “experience”, however you prefer to think about it. Some people are writing their first line of code today with LLM assistance; I wrote mine without an internet connection. Google came later, as did Stack Overflow, as did ubiquitous small open source projects. In each case I could make a conscious decision about the extent to which I would use the new thing—often substantially, but rarely as much as those who arrived after the new-to-me thing was commonplace.

My path isn’t optimal, it’s just the one I happened to take. For somebody learning today, it may well be the case that prudent use of an in-editor LLM for rapid Q&A will expedite their journey. Jeremy Howard’s new course How To Solve It With Code seems to be an earnest exploration of this possibility, so I look forward to working through it, and to seeing similar efforts.

Will LLMs substantially change the industry?

Software engineering tends to take its greatest strides from one major abstraction to another. These are innovations like the stored program, the compiler, the relational database system, and the process abstraction in operating systems. These dramatically changed the kind of work we could do—not to mention the constitution of the labor force—by packing a great deal of functionality behind a high leverage interface that could still be pierced as required.

My favorite example is Fortran, or more broadly “high level” (higher than assembly) programming languages. When Fortran, COBOL and Lisp all showed up in the late 50s, it was clear that programming would never be the same again: programmers would no longer require an intimate understanding of the machine; they could think in terms of the high level language (“formulas” in the case of Fortran) and have that be compiled or interpreted. As early as 1954, Fortran’s lead designer John Backus imagined that this should virtually eliminate coding and debugging, and he wasn’t entirely wrong. Coding as he knew it is now extremely rare. Very few people write much assembly.

But these languages and there descendants were very thoughtfully designed, the compilers are understandable without strictly needing to be understood, the disassembly can still be generated and stepped through one instruction at a time if necessary. These are phenomenal abstractions.

I would ask LLM hyperenthusiasts to question the degree to which LLMs are good abstractions, or can be incorporated into good abstractions in the future. Perhaps we end up with programming languages designed as LLM targets, where it’s easy to prompt engineer in front of this abstraction, and to debug deterministically behind it when needed. But these have not yet been invented, and it remains to be seen whether such a target language could be mostly ignored in a way comparable to a compiler’s output or intermediate representation.

If such abstractions don’t eventuate, LLMs may end up only as tools, with an impact comparable to IDEs or Stack Overflow, but well short of the hype.

Teach Yourself CS updates

I’m planning a major round of updates to teachyourselfcs.com around the end of the year, focused on clarifying context for topics and resources that come across as “pre-requisites” like SICP for programming, or broadly the topic of mathematics for computer science. A major failure mode I see with those using the curriculum is that they hit a roadblock on a topic they see as necessary for one they’re actually interested in, and give up rather than either persevering or skipping ahead.

I’m also planning to refresh some of the specific suggestions. On my radar are Andy Pavlo’s intro databases course, John Denero’s Composing Programs for introductory programming, the Kurose and Ross lecture recordings for networking, and a few others. If you have a favorite resource not yet on Teach Yourself CS, or another suggestion, this is a good time to send it through!

I’d also like to add some more self-guided project suggestions for those not able to commit to CS Primer. This is the biggest failure mode I see with Teach Yourself CS: grinding through textbooks and video lectures without consolidating that knowledge with projects and exercises.

CS Primer updates

My content production work on CS Primer is progressing decently, although slower than I’d like as usual. Since last newsletter I’ve cleaned up a number of existing courses, released an early version of Programming: Beyond the Basics and am progressively releasing the Relational Databases course, where the major project sequence is to write your own from scratch!

Jason’s MLE club

Former Bradfield student Jason Benn has been running a sabbatical/study club for machine learning engineers, meeting in person in San Francisco. Check it out, or see the podcast episode with Jason below for a little context.

Book recommendations

My favorite non-fiction book I read since last writing was Failure is Not an Option by Gene Kranz, flight director for many of the Mercury and Apollo missions including Apollo 11 and Apollo 13. It’s a fascinating view of the first era of space exploration, best paired with a pilgrimage to see the Saturn V at Kennedy Space Center. I speak about this a little with Charlie in a podcast recording, linked below.

My favorite fiction book over this period was Project Hail Mary by Andy Weir, an extremely fun read, with much of the engineering napkin math style of The Martian, but also some quite interesting exploration of communication with an alien life form.

Some honorable mention recent reads:

• Reentry, on the start of the (hopefully) second space age

• The Wright Brothers for clarity around how they actually managed to invent powered flight. Best paired with a visit to the National Air and Space Museum.

• Metals in The Service of Man covering just about the perfect scope of what I wanted to know about metallurgy

Podcast recordings

In case you need to fill the void, here are some podcast recordings since the last newsletter:

• When failure is not an option mentioned above

• Jason Benn’s path to ML engineering

• Making the easy things harder (with Madison Kanna)

Teaching kids to code

Thorsten Ball asked me to share a little about how I’m teaching my kids to code. We are in the early days but so far I’ve really liked CodeMonkey for its thoughtful skill progression and balance between substance and kid-friendly whimsy. It starts with Lightbot style icon-based programming puzzles (good for 3+ year olds) and progresses through Scratch style block coding, to short puzzles in CoffeeScript (don’t laugh, it’s great for young and old) and Python.

With this basic competency in Scratch, my 5 year old was then able to make a few games, mostly riffing off those in the book Coding Games in Scratch, generally with me my her side to debug as she went. It may not seem like “real learning” to copy 80% of a program from a book, but you may be surprised how many coders of my vintage first learned to code by tying in QBASIC games from a magazine, or using “view source”.

At this point my 3 year old is dabbling in CodeMonkey puzzles, mostly motivated by her big sister. She may have only put in a few hours, but she does have a basic intuition for loops and procedures in a way that’s typically achieved much more circuitously through the “computational thinking” offline games and activities typically pitched to this age range.

My 5 year old is currently doing simple Python programming, with a mix of the CodeMonkey challenges, a book Coding Projects in Python and of course a little direct tutoring from dad. For anybody who has taught their kids Python, I’d appreciate any tips for knocking off the sharp edges while using real tools, so that we can keep progressing. At the moment we are using VS Code for a little linting and tab completion, but I haven’t thought much about configuring it for a kid.

Overall the core of my pedagogy is trying to find the right challenge for that learner at that time. It’s rarely a good idea to dump information on somebody—particularly a kid!—but it’s also problematic to just provide an open-ended “learning environment” and hope for a good feedback loop. So while some kids can do ok with information-heavy resources at one end or a constructionist playground like Scratch at the other, I think most are going to thrive on something in the middle: a sequence of challenges, or somewhat structured mini projects, or a mix.

Synthesis “AI tutor” is not bad

I mostly avoided this product because I can’t imagine a good “AI tutor” at our current level of AI capability. Thankfully it appears that the use of AI is well restrained, and the positioning is mostly for hype. In practice, there’s a fairly thoughtful sequence of mostly multiple choice prompts around interactive widgets, to teach topics like number systems and arithmetic. The prompts are somewhat responsive—mostly by way of decision trees, and sometimes with a text field allowing for natural language input. My 5 year old quite enjoys it, and is currently using it alongside her long running favorites Beast Academy and Matific.

This is a good use of current capabilities in my view, and allows for a substantially better experience than less interactive substitutes such as Khan Academy and the like. It is of course no replacement for a good tutor, but again, it’s right there, and cheap, and can be done with my loose supervision rather than full attention.

Overall I quite like this approach of focusing on core instructional design, but using LLMs (I presume) to allow for higher interactivity and reduced friction. I previously explored a similar design—with mostly pre-scripted responses, but LLM-based classification of user input and some highly targeted LLM generated feedback—in unpublished prototypes, one for geometric construction and another for a software library. I didn’t continue these mostly for lack of time, but I think it would be particularly great for learning to code, or similar scenarios that involve basic guided feedback on an artefact. If you are interested in working on something like this, let me know.

I worked briefly at Counsyl, a genetic testing company founded by Ramji and Balaji Srinivasan. I was burnt out before starting so mostly wasted the opportunity to learn from these immensely capable brothers, but I did pick up a thing or two worth sharing.

From Ramji, I learnt “list, rank, iterate” the world’s most effective operational strategy that hardly anyone uses. Balaji explains it in this Conversation with Tyler.

I also observed a master CEO at work: somehow Ramji could identify the company’s biggest problem, rebuild himself into just the leader who could solve it, then do it again every 6 months until he sold the company to Myriad Genetics for $375 million.

You can’t build a biotech company through operational effectiveness alone, though. You also need a mad scientist! Thankfully there was one in the family: Counsyl started thanks to a genetic testing technique invented by Balaji combining his expertise in genomics, chemical and electrical engineering, software and statistics.

I had assumed he’d been taught what he knew, at Stanford, a place he talks of very fondly. I figured he’d encourage others to do the same. Coming into a genomics company with little background in science, I proposed to him that I might attend his alma mater in the evenings to fill some gaps and maybe pursue a masters degree.

He told me that would be a colossal waste of time. Why not just teach myself? I was surprised to learn that this is what he’d done himself from a young age, achieving mastery of multiple challenging fields, and that he loved Stanford for the ethos more than the education. From Balaji, I learnt how much it is possible to teach oneself.

This is how he suggested I go about things: get a textbook and open it up to the problems. Do them. You might be surprised, he said, how much you can figure out yourself. Only when I got stuck should I consider reading the prose. Keep solving the problems maniacally, and one set of techniques will unlock another.

I was too distracted to fully heed his advice at the time, but I’ve carried it with me, and it’s the primary way I approach self-teaching now. I seek out textbooks with thoughtful problems, or that directly help me solve one that’s been bugging me. I ignore any unsolicited explanatory material, like a MOOC that “looks interesting” or a book on a topic I “should learn”. If I encounter a problem set, challenge, project idea or just a juicy question, I will file it away for later.

Balaji was particularly fond of a series of books Schaum’s Outlines. This surprised me, as they’re basically cram books marketed at overwhelmed college students. Their authors are qualified but not particularly distinguished, and the prose is generally uninspired, but each has hundreds of worked problems! If you agree that learning organic chemistry, say, is ultimately about being able to solve problems involving organic chemistry, then this kind of practice is surely more effective than watching a lecture.

I can’t vouch for the quality of any given Schaum’s book. In fact, the corresponding Schaum for any given topic is unlikely to be the absolute best book: One Thousand Exercises in Probability is probably better than the Schaum’s equivalent. But the Schaum’s books are cheap, easy to find, and often a fast track to competency. Perhaps they will work as well for you as they did for Balaji.

This eventually changed with the invention of the blast furnace, but unfortunately the resulting cast iron was too brittle for many applications. It wasn’t until the development of the Bessemer Process in the 1850s that we could control the carbon content of molten iron reliably enough to make steel.

What I find fascinating about this history is that for 3,000 years, ironsmiths were just banging away at slag, oblivious to the fact that we would eventually find a much better method. This is the crazy thing about technology. There can be an obvious and universal way of doing something, until it isn’t. Just imagine yourself hammering iron blooms in the tradition of generations of smiths before you… what would prompt you to even contemplate doing things differently?

Better technique, better results

I like to ask people what they think is “the problem with education”. It’s a fun way to get a glimpse of their general worldview. Perhaps it’s an unfair question but everybody seems to take for granted that there is in fact a problem, and typically just one big one.

Depending on who I ask, the problem is that there’s too little school choice, or too much of it, or that the elites are too progressive or too conservative, or that we need to pay teachers more or dismantle their unions. Like I said, it’s a fun question.

The most interesting answers to me are technological, in the broad sense of “technology” as a repeatable technique, like pumping air into a blast furnace. I’m generally not swayed by the political arguments, because education is so broadly ineffective. We tend to be bad at teaching things both within and outside of traditional systems, whether or not the state in question has a voucher system, and so on.

The technological answers are exciting though because they suggest that we may be just one idea away from a breakthrough. We may be able to get better results simply by doing a better job.

Have you tried not blurting out the answer?

An acquaintance of mine runs a coding interview platform. When LLMs started taking off, they trialed an LLM based interviewer, but it failed somewhat comically: if a candidate showed any sign of struggle, the LLM would just blurt out the answer.

This is a major barrier to LLM use in education, too. We’d like AI tutors to debug student misconceptions and ask well-calibrated questions, like an expert tennis coach feeding balls with just the right amount of spin. Unfortunately they tend to prefer reeling off explanations, so the hard work falls on the user.

In fairness to AI tutors, human educators rarely do much better! Most lecturers just lecture. Most textbooks just explain. Standardized curricula lay out bodies of knowledge as a sequence of foregone conclusions: this is how you manipulate algebraic symbols, this is how you factor polynomials, this is how you differentiate them, and so on. Exercises tend to just check whether you were listening. Factor the given polynomials.

This is the standard way of teaching: we want students to know X so we tell them X. It seems like it should work, and in a way it does, like whacking iron blooms repeatedly with a hammer. It is true that some people manage to learn mathematics this way, but it is rare. If you don’t believe me, try factoring a polynomial.

Have I mentioned nand2tetris yet?

There are some educational resources that I can’t help but recommend over and over again, to the point where students joke about having them on their bingo cards. These are things like nand2tetris, or Silicon Zeroes and the Zachtronics games. Among programming books, those like The Little Schemer, with a thousand tiny questions. Among textbooks, those that are project oriented, like Coding the Matrix. Perhaps a dozen sites full of problems: Project Euler, exercism, cryptopals. Outside of computing, I keep raving about Caveman Chemistry, which is like nand2tetris for the physical world.

What makes all of these resources wonderful is that they don’t blurt out the answers. Each is a well designed set of challenges, problems, questions: ultimately, learning stimulus. They are fun, and in working through them you learn their subject matter enjoyably and authentically. The Little Schemer, is full of cartoons and starts very gently, but by the end you will have implemented a meta-circular evaluator.

Sadly, these resources can be hard to produce and challenging to cater to a broad audience. Many people find Project Euler too mathy, for instance, and give up after a problem or two, but one non-programmer friend to whom I recommended it disappeared for a few weeks and remerged as a highly capable coder. These challenges of problem-based education resources I see as akin to the brittleness of cast iron: something to be aware of and work around, through further innovation.

CS Primer is Generally Available

All of this is my uniquely convoluted way of announcing that CS Primer is now generally available, after over a year of paid beta. I am aware that writing this at the end of a rambling article is not optimal in terms of marketing, but I wanted to give you better context on why I designed it this way.

The biggest problem with how we teach, in my opinion, is that we use the blunt force tool of simply telling students what we think they should know. This is not nearly as effective, in my experience, as setting up a sequence of well calibrated challenges: at least questions, but ideally exercises or projects that are authentic to the topic being studied. For systems focused computer science, this means programming problems, so these are the backbone of CS Primer. I have included my best problems from teaching these topics in person and seeing what students find most rewarding.

At the same time, I’m well aware of the hazards of building an entire CS curriculum around programming problems. Most importantly, nobody should get stuck on a problem and left to flail. My attempt to address these challenges is to include full video walkthroughs to all of the problems, structured with hints and stretch goals to allow students to calibrate the difficulty for themselves. When progress on a problem requires a piece of conceptual understanding, that’s when I provide a short explanation, as a JIT supplement to the walkthrough. If that’s not enough, I’m available via Discord and office hours.

I’ve now given this treatment to four courses: algorithms and data structures, computer architecture, operating systems and networking. For most software engineers this will take a year or two to work through in earnest: there are over 100 hours of recorded content but of course it’s more about the time that you spend solving the problems, than watching me do the same. I’m still working on a few remaining courses but I’m happy with what’s there and excited to release general availability! There are many possible ways to learn computer science but this is the most effective way I could imagine for those who can already code, while still being self-paced and so affordable to students worldwide.

If you choose to sign up, I hope you find it challenging and rewarding. Good luck with the problems!

You may be wondering, what is the transition that brought us here? I'm not referring to existence or anything, I just mean this Substack. Was your email address sold to a surprisingly long-winded spammer? Did you sign up for it in a transitional moment of weakness? No, loyal reader, it was the kind of transition known as a migration: I was paying a substantial amount of money to maintain a mailing list which I emailed infrequently, whereas Substack figured out how to store 11,154 rows in a database for much less than $50/month, so here we are. You may have signed up via teachyourselfcs.com or the Bradfield website. If you'd rather not receive these, that's totally fine by me! There's a link to unsubscribe at the bottom.

Career transitions

My job since 2015 has been to teach computer science, mostly to those who transitioned into software engineering from another field.

Software as an industry is fantastic and dysfunctional mostly for the same reason: we don’t have a professional body to maintain any kind of standard of professional practice. We're simply not a profession. If you want to be a civil engineer in the US, you need to complete an accredited degree program then pursue certification overseen by the American Society of Civil Engineers. To be a software engineer, you need to grind Leetcode until you can performatively solve "medium" difficulty dynamic programming problems. Or, you can just start calling yourself a software engineer, maybe make a portfolio website that says "I'm not a charlatan, I even did a Rust tutorial", and start sending out resumes.

Sometimes I wonder if our software systems might be of a higher standard if we had more gatekeeping. But then I think about doctors and lawyers and public school teachers and snap out of my daydream. Even if we could improve software quality a little by decertifying the occasional reprobate (I see you "serverless" folk avoiding eye contact right now) it would undermine what's really great about what we do: practically anybody can learn to write software, given enough motivation, and practically any of those people can bring a lot of value to the world, or at least their families and 401K balances, by doing so.

I was never in the business of helping people with this particular transition, but I was fortunate to see it happening. While I was CTO of Vida Health, my friend Myles was teaching at one of the first coding bootcamps, and it was he who convinced me that a lot of talented people were entering the industry this way. I hired one of them (Omar), helped him learn a little computer science on the job, then quit to spend the next 8 years helping other non-traditionally educated engineers do the same. I've sometimes described this as joining my students for a few steps on their journeys. I would love to take credit for their sometimes spectacular success, but if anything I show up right after they've done the hard work.

Your transitions

I'm constantly inspired by the leaps of faith that my students have taken. Omar was an ophthalmologist, in Spain, with kids. While sure, he's the only former Spanish ophthalmologist dad turned Senior Staff Software Engineer I'm aware of, many others left equally comfortable career tracks to find their respective callings in software. Among those who worked their way into web development, many ventured out once more to challenging specialty areas like performance engineering, platform and infrastructure work, security, even quantum computing.

Perhaps there's a degree of survivorship bias, but I struggle to think of cases where somebody took too much of a risk, quit their job too soon, or was too ambitious with their next set of career goals. When somebody asks me for advice on a potential transition like this, I'm always honest about what it might take: are they ready, for instance, to learn the math required for that machine learning researcher role they’re interested in? But I also point out: even when you fail to achieve your goal, you are likely to have ended up somewhere better than where you started. Very few people invest in themselves once they have established, well-paying careers. Almost nobody studies in their 30s. If you take some time off to work through some textbooks or tinker with a programming project or two, you're almost certain to end up ahead.

So why are career risks so underrated, particularly in computing? Frankly, I think many of us feel like things are too good to be true, that we don't want to risk our enviable, cushy L-whatever positions for something indulgent like the thrill of a major technical challenge.

But life is short. You should learn about what excites you and direct your productive energy to where it can have the most impact.

My transition

After 8 years of running the Bradfield School of Computer Science, I have decided to send it a SIGTSTP. While I may SIGCONT it at some point, my current plan is to stop teaching live classes once the current cohort graduates in a few months.

It feels strange to stop something that’s running so well: each year the curriculum is better, the brand is stronger, we teach more effectively and students get more value. So why stop now?

Simply put, I feel acutely aware that life is short, and would like to learn about what excites me and direct my productive energy where I can have the most impact. Most notably, CS Primer is my ongoing attempt to adapt all of the best parts of live Bradfield classes to a more affordable self-paced format. My goal with Bradfield was to be the best place for self-taught engineers to learn foundational, systems-focused computer science. While many students felt that Bradfield achieved precisely that, the structure of live small group instruction would always make it inaccessible to many of the people who would benefit the most. So I would like to dedicate myself to CS Primer, and then to other educational projects that could operate at a broader scale.

That said, it has been a tremendous privilege to share a few steps on the journey of each individual student we have had. A little over 1,000 of you took our classes; many of you investing hundreds of hours of your time. I have always felt honoured by your commitment, and I hope we did it justice. The hardest thing about this decision is knowing how much I’ll miss being in the classroom with you all, having fun while we learn a little computer science together.

To the instructors who taught alongside me over the years: Brian Spiering, Haseeb Qureshi, Tim Olshansky, Tom Alcorn, Tom Ballinger, Tyler Bettilyon, Xavier Shay… a heartfelt thank you for your commitment to the cause, and for everything you did for your students. An extra special thank you to Myles Byrne, without whom this all simply wouldn’t have started, and to Elliott Jin, my fellow traveler to the end. I am forever grateful.

Awkward transitions

Oh, umm, while you’re here:

• If you’re a former Bradfield student and would like to leave a short reflection on the website in memoriam, please write it here. I thought this would be a nice way to leave the website, and also leave a good impression with any recruiters or hiring managers who see Bradfield on a resume.

• I'll be in the US in June, July and August. My final Bradfield class will be in person, and likely open to everybody. I'll also host an alumni trip to the Computer History Museum in Mountain View. Stay tuned here for updates, if you'd like to join.

• Charlie and I did a number of podcast interviews since my last email, including with some amazing folk like Thorsten Ball, Zach Latta, Brandon Hendrickson, Brit Cruise, Linda Liukas, Jimmy Soni and Jon Gertner, and the man, myth, legend and Bradfield alum Richie Artoul

• Speaking of Richie, he announced their $20M fundraise and plans for world domination with WarpStream. Go, man, go!

• I wrote a little thing highlighting some project ideas that I'd be excited to collaborate on if somebody came to me with a lot of their own energy.

• For those who haven't checked in on CS Primer for a while, I now have almost all the content up for Computer Systems, Computer Networking, Operating Systems and Algorithms and Data Structures. This is around the halfway point for content, so I'll be launching the general availability release soon after somehow a year in paid beta!

• Home schooling is going great. My 3yo is learning to read now in much the same way that my 5yo did at the same age, and I'm having a lot of fun doing science projects with both of them. I highly recommend home schooling if you can, even if it's just an hour in the morning or afternoon around school. If you're taking a more active role in your kids education, and would like to join a little Discord server of others, let me know.

• The best nonfiction book I've read recently was Origins by Lewis Dartnell, the author of one of my all time favorites The Knowledge. On the fiction side, it was Last and First Men, a mind altering work spanning 2 billion years. Both very highly recommended.

• I'm spending my study energy at the moment on Molecular Biology of the Cell, mostly out of long suppressed fascination, but also with half a notion that I might do some work in computational biology some day. If you are or know of somebody who might give me a little mentorship in this area, please let me know! 🙏

That's all for now. Thank you everyone 🫡. Back to your irregular schedule of computer science education and tech news that matters, after this.