> but these days I think the much more important skill is designing systems.

It is hard to design systems if you don't have the perspective of implementing them. Yes, you move up the value chain to designing things, no, but no, you don't get to skip gaining experience lower down the value chain.

> What computer science doesn’t teach you is how memory caching works in CPUs.

That was literally my first quarter in my CS undergrad 30 years ago, the old Hennessy and Patterson book, which I believe is still used today. Are things so different now?

> The much more common problems I have is how to deal with fault tolerance, correctness in distributed locks and queues, and system scalability.

All of that was covered in my CS undergrad, I wasn't even in a fancy computer engineering/EE background.

Well CS and software dev in trenches moved a bit.

There are still jobs where people write frameworks, database engines or version control tools. Those jobs require heavy CS and algorithms, data structures day to day. But there are less of those jobs nowadays as no one is implementing db engine for their app they just use Postgres.

Other jobs that is vast majority is dealing with implementing business logic. Using database with understanding how it works in details is of course going to produce better outcomes. Yet one still can produce great amount of working software without knowing how indexes are stored on disk.

Also a lot of CS graduates fell into a trap where they think their job is to write a framework - where in reality they should just use frameworks and implement business logic- while using CS background to fully understand frameworks already existing.

>What computer science doesn’t teach you is how memory caching works in CPUs. Your fancy graph algorithm may have good runtime complexity but it completely hoses the CPU cache and you may have been able to go faster with an array with good cache usage.

Traditionally, the field of databases is largely about solving algorithm problems in the scenario where you have much more data that can fit in memory. Data exists on disk as "pages", you have a fixed number of "page slots" in RAM. Moving pages from disk to RAM or RAM to disk is slow, so you want to do as little of that as you can. This makes trivial problems interesting -- e.g. there's no notion of a 'join' in classic computer science because it's too trivial to bother naming.

We're used to thinking of the study of algorithms as a sort of pure essence, but one could argue that algorithmic efficiency is only meaningful in a particular data and hardware context. That's part of what keeps our jobs interesting, I guess -- otherwise algorithm expertise wouldn't be as useful, since you could just apply libraries/cookbook solutions everywhere.

True. However I find that most junior and even experienced programmers struggle with tactical level coding. I'm really suffering with this right now because the small component I'm tasked with making a small change to is annoyingly stateful and deals with 2 abstractions at once. (It processes files and uses the file system and database to store it's state). I'm shocked how badly it has been thought out. I've spent days trying to avoid doing what has gone before, bits bolted on that make it even more difficult to understand. It really seems that pull request culture has just led to any old crap being approved because no one has the band width to think deeply about the actual code. Bring back in person code reviews !

Have you seen

"Software Design for Flexibility: How to Avoid Programming Yourself into a Corner" by Chris Hanson and Gerald Jay Sussman

It's from 2021.

Scouring SICP cannot imbue the student with mechanical sympathy any more than poring over analysis of Coltrane makes me a saxophonist.

Nevertheless. It must be done. Theory and practice.

I find books like SICP interesting and not very useful. I love reading them because I like this stuff, but I don’t get to apply their teachings in real world software. It’s a problem because naturally I want to spend my time reading these kind of books, but if I do that I would be jobless. I need to divide my time between reading pearls like SICP and boring Kafka/Postgres/Golang/K8s/AWS documentation.

You're in luck. Part 5 of the book is about building a virtual machine to run lisp simalated at the register lelvel: https://mitp-content-server.mit.edu/books/content/sectbyfn/b...

Writing a network between n such machines is left as an exercise to the reader.

> I think the much more important skill is designing systems

Systems engineering is a seperate discipline in engineering fields, same applies to CS I would think, specifically if you look at what most developers do as computer engineering it would only make sense that there is forward progression within the discipline that builds on the foundations tought in most CS courses.

Regarding CS not teaching caching, well any course writer in a CS program that doesn't at least touch on that should feel like they failed their students, it is really something fundamental that should be tought pretty soon.

I feel like profiling should be something that is tought somewhere in a CS program, even a half semester course can dramatically improve peoples understanding of what the machine is actually doing.

On that note, even if you never intend to use the language, watch some C++ conference talks, almost every year at every conference thwre is a talk discussing performance.

The right book for the right problem. SICP isn't meant to teach you how to tackle fault-tolerance in a complex distributed system. Here is a textbook that talks about distributed systems (van Steen and Tannenbaum):

https://www.amazon.ca/Distributed-Systems-Maarten-van-Steen/...

> these days I think the much more important skill is designing systems

That's true, but that doesn't mean that there is no value in having an understanding of how established technology works under the hood.

> What computer science doesn’t teach you is how memory caching works in CPUs.

That is also a very good point. There is a lot of daylight between the lambda calculus and real systems.

> big O notation and runtime complexity doesn’t matter the majority of the time and you can solve most problems with arrays

I have the exact opposite experience.

Software comes out best if you always ensure to use an approach with sensible runtime complexity, and only make trade-offs towards cache-friendly-worse-O implementations where you benchmarked thoroughly.

Most cases where I encounter mega slow programs are because somebody put in something quadratic instead of using a simple, standard O(n logn) solution.

Check out https://www.tumblr.com/accidentallyquadratic for many examples.

> What computer science doesn’t teach you is how memory caching works in CPUs. Your fancy graph algorithm may have good runtime complexity but it completely hoses the CPU cache and you may have been able to go faster with an array with good cache usage.

Computer architecture and organization should teach this, no?

There are still innumerable people writing standalone programs, single-purpose embedded systems, independent components and libraries, etc

The industry has expanded to include a lot of large-scale distributed cloud projects (often where we might have expected mainframes and cobol before), with many of today's largest employers doing most of their work there, but none of that other stuff really went away. It's still being done every day.

You need a book for what you're doing, and not every book is going to be that. Apparently, SICP is not it. I possess and have read many books, and only some small number of them are applicable to the projects I'm working on at any time.

They don't compete with each other, they complement each other.

> What computer science doesn’t teach you is how memory caching works in CPUs.

Yes it can, and there are tons of papers about data structures to use in various scenarios to handle not just L1, L2, L3, but also NUMA. Sure, this isn’t in SICP, but claiming CS as a field completely ignores how memory works is incorrect.

If you knew how to design programs you could run it all on a single box and wouldn’t have to design “systems.”

I’m being slightly facetious, but only slightly. If you really think everything is solvable with arrays, you are not going to scale well and of course you’re going to need to throw a lot more hardware at the problem.

Since you emphasize designing systems over just programs, do you have any go-to resources or references ?

My CS degree covered hardware architecture, disk design, OS design, and also system design, alongside algorithms, data-structures, various programming language and paradigm, etc.

> I don’t know if distributed systems is consider part of “Computer Science"

It surely was part of my Informatics Engineering degree, with Tanenbaum book being one of the required reads.

I think abstraction layers exist at the system design level as well. Many of the things SICP teaches apply there as well.

Do you have any distributed systems design books to recommend?

I think the classic CLR text is great but, yeah, caches through quite a monkey wrench into naive big O analysis.

Still, I think the time investment to learn algos and data structures isn't too much of a burden.

For anyone wishing to try: the maintainers of MIT Scheme no longer provide a .dmg but you can download and build the x86_64 version of MIT Scheme. The current release (v12.1) works on a Mac running Sequoia with Intel CPU or on Apple silicon via Rosetta. But the native code compiler (not necessary for SICP AFAIK) is a little broken. (Anecdotally it worked on macOS prior to Monterey, so maybe an Apple-supplied dependency changed. Haven't tracked down the issue.)

All of that is to say: if you do not need MIT Scheme and don't want to fuss with compiling it, then Racket might be a better way to go.

Just as a data point, I'd recommend going through it in Racket, which I believe has an explicit SICP mode. I went through it in GNU Guile and it was a pain because there were some minor syntactic differences between Guile and MIT Scheme.

The texinfo version was I believe the source for the really nice HTML5 version if you want to read it in a browser, but with nice formatting that the MIT original version: https://sarabander.github.io/sicp/

one thing to note is that the second chapter's "picture language" is not supported in MIT Scheme in 2024. There used to be a package but it's like 2 decades out of maintenance. In Dr. Racket however, there is a package specifically for working through those problems.

Dr Racket has SICP and HTDP as a teaching pack.

The Kabbalah joke gets me every time.

Thanks for sharing this. It's crazy to see videos like this and think how these people had no idea what was coming in the future.

Oh and here I thought you were talking about Elements of Programming by Stepanov and McJones which tbh I'd give the same recommendation/review.

https://elementsofprogramming.com/

They do that because their wish is performance and naturalness.

You may accidentally wish something you don’t yet know the true nature of, and this will create a fragile mess at the bottom. It usually does, cause algorithmic nature of things is rarely intuitive. Starting from the bottom is like starting from quarks that you have rather than from “I want magic to exist”. Well it does not. You reach the bottom and there’s quarks instead of magicules and you’ve lost all context clues on the way which could help to convert between two physics.

Both approaches have their use, because sometimes you have to be bold with your wishes to solve a deep problem. But personally I prefer magic to be packed into the before-topmost layer. I.e. build from the bottom up, and then, just before the business logic, create a convenience magic layer that translates to/from business speak. It becomes adjustable and doesn’t induce a tangled mess all the way down.

Interestingly, Dr. Donald Knuth used pretty much that approach when writing TeX --- he started by writing out the sort of formatting/tagging which seemed appropriate, then theorizing about the sort of programming which would be appropriate for markup (hence macros), then worked on the implementation.

I've been trying a similar thing for my own effort to create a library for modeling G-code in OpenSCAD --- hopefully with the recent re-write in "pure" OpenPythonSCAD it will become something usable.

The Smalltalk world has great support for this, through coding in the debugger. You should try Pharo.

You might be interested in a 1987 article titled "A Critique of Abelson and Sussman or Why Calculating is Better than Scheming" (https://dl.acm.org/doi/10.1145/24697.24706), where the author advocates the use of KRC or Miranda as alternatives to Scheme. I don't know much about KRC, but Miranda is a statically-typed functional programming language that influenced Haskell.

SICP isn't a book about Lisp, however it uses some of Lisp's unique properties to demonstrate important concepts that other languages can't easily replicate. A book that's meant to be similar to SICP that doesn't use Scheme or Lisp would not be anything like SICP, or at least not teach the same things. Haskell and ML are in my experience much harder to understand than Scheme, so I'm wondering what your difficulty is?

I really wanted to like SICP and I probably would have if I read it 15 years ago. I started reading it last month and I found it to be too broad. It covers too much interesting mathematical principles and then jumps to the next one right when it starts to get interesting. In other words, it's too shallow.

It probably doesn't help that I've seen many courses/documents that are (in hindsight) derivatives from SICP, so I have the nagging thought "not this again" when a topic is introduced in SICP.

I can identify with that - Lisp throws me off (because I’m not smart enough). But I ended up forcing myself to work through it and learned a tremendous amount because I’m not smart enough to work with a lisp. It felt like I spent so much time just reading through the code that I ended up learning more than I would in a language I’m comfortable with.

There is a Python version of SICP. I have never worked through it or even given it more than a cursory scan so this is not an endorsement more just a link to prove it exists:

https://wizardforcel.gitbooks.io/sicp-in-python/content/0.ht...

"Functional Programming in Scala" aka "Red Book of Scala" is a the one that IMO teaches to think the same way as SICP, while using a typed language. The books stand next to each other on my bookshelf, definitely worth reading.

I don't understand why Lisp throws you off. I only read SICP after I became proficient in Haskell and it is just fine.

"Concepts, Techniques, and Models of Computer Programming" by Peter Van Roy and Seif Haridi is my favourite.

i thought berkeley was using a modified version of the book using python a few years back

yes, i like ML (well.. ocaml) and bounced off SICP for the same reason. It was actually SICM that made me come back and stick with it, the ideas were just too interesting (whereas for SICP it was a lot of ideas I was already familiar with)

I think there is a Python version if that floats your boat

One of the best programming classes I had in college was a comparative languages course where multiple languages were covered, each in two week or so blocks.

It’s a much, much denser successor to sicp. I hadn’t succeeded in self-studying with it despite strong lisp/scheme chops and strong affinity for sicp.

And the Python version: http://www.composingprograms.com/

Spelling correction on the second author's last name: Kaiser

I'll second that recommendation, it's a bit like a gentle mixture between SICP and TAOCP. A very enjoyable read.

I concur, I am learning from it now…

I think this was mentioned in a Paul Graham essay from many years ago, but half the battle in tackling a new area of study is finding out what the best books (or papers) are. There aren’t that many, and yet it’s hard to know what they are if you aren’t already well-versed in the field.

Just so we're clear, this is a "beginner programming book" that has you create a scheme interpreter, then a register machine simulator, then a compiler out of your interpreter that will then have its compiled code run on the register machine simulator, by the final chapter.

This is probably the part where you'd step up and post a link to your repo with solutions to the exercises to back up your talk, but generally I only see this sort of casual dismissal from people who haven't actually worked through the book.

I don’t understand this comment. If you master the material you know more than 90% of engineers in the field.

My same experience. For much of the rest of the cs curriculum I felt like we had already to some extent covered the main ideas in 61a with sicp.

I think in the context of the book 'procedural epistemology' encompasses all programming, not just what you'd call procedural programming.

You're doing the exercises in Elixir and not Scheme then?

> In other words, I say "SICP considered harmful" because thrusting it upon an eager newcomer as a trusted neutral guide to beginner coding (without offering any counterpoint) could set them back by a decade, filling their head with "functional object oriented programming" concepts that don't translate well to industry or CS.

I'd counter that by saying it would set them forward by a decade (compared to people who don't know these techniques). Knowing advanced techniques doesn't mean trying to shoehorn them into every run of the mill problem you encounter in the industry. But if you encounter a gnarly problem where some advanced techniques will help you out, you'll sure be glad you learnt them.

I would say that after 20+ years career OO plus functional elements have been a good default. Maybe I missed opportunities where symbolic computations were important and limited myself with languages like C++ that prefer OO/functional approaches... I can't go back and try something else to know =)

I might agree with your hot take in sense that leaving choice is important though.

I recall that when MIT stopped teaching with SICP, one of the main claims was that programming now is often not about thinking abstractions through from first principles, and creating some isolated gem of composing definitions.

Which is a category mistake that they actually address in the lectures. SICP is not a programming course, it’s a computer science course. Computer science is not about computers, let alone programming, just as geometry is not about surveying instruments and astronomy is not about telescopes.

When they stopped teaching SICP — in response to the pressure to teach more modern tools — they abandoned their scientific principles to satisfy commercial concerns. They stopped teaching computer science and became a vocational school for the tech industry.

I've witnessed how abandoning first principles undermines the evolution of a system. If our mental model of a system is not formalized into first principles (i.e. a high-level specification), then successive generations of engineers will have to re-learn those principles through trial-and-error. They'll introduce mutations and dependencies between the mutations-- and when they leave, the next generation of maintainers will repeat the process. Generations of mutations eventually create a brittle, calcified creature of a system which people fear to touch with a ten foot poll.

I imagine people who were taught SICP would be more respectful, if not inclined, towards a formal articulation of a system's principles.

This philosophy is described in depth in the original 1985 article https://gwern.net/doc/cs/algorithm/1985-naur.pdf and in more accessible language in https://www.baldurbjarnason.com/2022/theory-building/. You can also observe engineers opposing/misunderstanding the need for specification in https://news.ycombinator.com/item?id=42114874

I think just like traditional engineers have to learn physics, computer people should learn these fundamentals for exactly the reason you outline.

Then, when you hit the job market, you learn the ecosystem of what other engineers have built and you work in that context.

In this way, you can eventually reach extreme productivity. Just look at humanity's GDP over the last 200 years.

Scheme as basically an implementation of the untyped lambda calculus will eternally be a good frame work to think about the problems of computation in.

In the more practical area Racket (the most modern Scheme) has basically any practical functionality you would want, while amazingly remaining a platform for an incredible amount of experimentation in computation and programming language theory.

But SICP is a book that is for people interested in the study of computation what programming languages can be. If you're worried about getting a job in software it won't be all that useful, but it will remain a classic for anyone interested in engaging in creating the future of software.

Sure, SICP is not a good book for people wanting to do rote learning, imitation, 'best practice' while ignoring the history.

It's for people that would like to learn rather advanced programming techniques and foundational ideas in computer science.