Oh my god, same. This literally catches bugs with a smoking gun in their hand in a way that's completely impossible with printf. I'd upvote this 100 times if I could.

windbg used to offer scripting capabilities that teams could use to trigger validation of any number of internal data structures essentially at every breakpoint or watchpoint trigger. it was a tremendous way to detect subtle state corruption. and sharing scripts across teams was also a way to share knowledge of a complex binary that was often not encoded in asserts or other aspects of the codebase.

From the same toolbox: expression watch. Set a watch on the invariant being violated (say "bufpos < buflen") and get a breakpoint the moment it changes.

Especially with combined with reverse-execution in rr or UndoDB!

Is there somewhere where this approach is described in more detail?

I am the author of the posted flamebait. I agree.

I use IPython / JShell REPLs often when the code is not finished and I have to call a random function without entrypoint.

In fact its possible to jump to the graphical debugger from the Python REPL when running locally. PyCharm has this feature natively. In VSCode you can use a simple workaround like this: https://mahesh-hegde.github.io/posts/vscode-ipython-debuggin...

Print debugging is historical / offline debugging, just ad-hoc instead of systemic.

The ”debug” package on npm is something in between, as it requires inserting debug statements but they are hidden from output unless an envvar like DEBUG=scope.subscope.*,otherscope is used.

I've loved working with rr! Unfortunately the most recent project I've been contributing to breaks it (honestly it might just be Ubuntu, as it works on my arch install, but doesn't work when deployed where I need to test it).

I frequently use the go debugger to debug concurrent go routines. I haven’t found it any different than single threaded debugging.

I simply use conditional break points to break when whatever go routine happens to be working on the struct I care about.

Is there more to the issue?

lol, agree to disagree here. While the interface to gdb is annoying, there are many gui frontend alternatives.

VS, on the other hand, gets worse with every release. It is intolerably slow and buggy at this point. It used to be a fantastic piece of software, and is now a fantastic pile of shit.

> I’m going to reach for the tool that always works vs. sometimes works.

This is only logical if you're limited to one tool. Would you never buy a power tool because sometimes the power goes out and a hand tool is your only choice?

but can you go back in the stack and inspect the variables and related functions there in print debugging?

Depending on the language or setup debuggers can be really crappy. I think people here would just flee away and go find a better fitting stack, but for more pragmatic workers they'll just learn to debug with the other tools (REPL, structured logging, APMs etc.)

I had a think about where I first learned to use a debugger. The combo of M$ making it easy for .NET and VB6 and working professionally and learning from others was key. Surprised it is less popular. Tests have made it less necessary perhaps BUT debugging a unit test is a killer move. You quickly get to the breakpoint and can tweak the scenario.

yeah, tons dont know they exist. But there's also a lot of people - new and veteran - who are just allergic to them, for various reasons.

Setting up a debugger is the very first thing i do when i start working with a new language, and always use it to explore the code on new projects.

This also applies to testing. So much legacy code out there that's untested.

printing is never the appropriate tool. You can make your debugger print something when that line of code is reached anyway and automatically continue if you want. So what's the point of pritntf? It's just less information and features.

I see it the exact other way around:

- everyday bugs, just put a breakpoint

- rare cases: add logging

By definition a rare case probably will rarely show up in my dev environment if it shows up at all, so the only way to find them is to add logging and look at the logs next time someone reports that same bug after the logging was added.

Something tells me your debugger is really hard to use, because otherwise why would you voluntarily choose to add and remove logging instead of just activating the debugger?

Rare 1% bugs practically require prints debugging because they are only going to appear only 6 times if you run the test 600 times. So you just run the test 600 times all at once, look at the logs of the 6 failed tests, and fix the bug. You don’t want to run the debugger 600 times in sequence.

The tricky race conditions are the ones you often don't see in the debugger, because stopping one thread makes the behavior deterministic. But that aside, for webapps I feel it's way easier to just set a breakpoint and stop to see a var's value instead of adding a print statement for it (just to find out that you also need to see the value of another var). So given you just always start in debugging mode, there's no downside if you have a good IDE.

> The rare ones show up maybe 1% of the time

Lucky you lol

What I've found is that as you chew through surface level issues, at one point all that's left is messy and tricky bugs.

Still have a vivid memory of moving a JS frontend to TS and just overnight losing all the "oh shucks" frontend bugs, being left with race conditions and friends.

Not to say you can't do print debugging with that (tracing is fancy print debugging!), but I've found that a project that has a lot of easy-to-debug issues tends to be at a certain level of maturity and as times goes on you start ripping your hair out way more

I used to agree with this, but then I realized that you can use trace points (aka non-suspending break points) in a debugger. These cover all the use cases of print statements with a few extra advantages:

- You can add new traces, or modify/disable existing ones at runtime without having to recompile and rerun your program.

- Once you've fixed the bug, you don't have to cleanup all the prints that you left around the codebase.

I know that there is a good reason for debugging with prints: The debugging experience of many languages suck. In that case I always use prints. But if I'm lucky to use a language with good debugging tooling (e.g Java/Kotlin + IntelliJ IDEA), there is zero chance to ever print for debugging.

I've had far better luck print debugging tricky race conditions than using a debugger.

The only language where I've found a debugger particularly useful for race condition debugging is go, where it's a lot easier to synthetically trigger race conditions in my experience.

Even print debugging is easier in a good debugger.

Print debugging in frontend JS/TS is literally just writing the statement "debugger;" and saving the file. JS, unlike supposedly better designed languages, is designed to support hot reloading so often times just saving the file will launch me into the debugger at the line of code in question.

I used to write C++, and setting up print statements, while easier than using LLDB, is still harder than that.

I still use print debugging, but only when the debugger fails me. It's still easier to write a series of console.log()s than to set up logging breakpoints. If only there was an equivalent to "debugger;" that supported log and continue.

Well, if you have a race condition, the debugger is likely to change the timing and alter the race, possibly hiding it altogether. Race conditions is where print is often more useful than the debugger.

Fully agree.

If I find myself using a debugger it’s usually one two things: - freshly written low level assembly code that isn’t working - basic userspace app crash (in C) where whipping out gdb is faster than adding prints and recompiling.

Even never needed a debugger for complex kernel drivers — just prints.

> the rare, tricky race conditions [...]. The rare ones show up maybe 1% of the time—they demand a debugger,

Interesting. I usually find those harder to debug with a debugger. Debuggers change the timing when stepping through, making the bug disappear. Do you have a cool trick for that? (Or a mundane trick, I'm not picky.)

It is also much much easier to fix all kinds of all other bugs stepping through code with the debugger.

I am in camp where 1% on the easy side of the curve can be efficiently fixed by print statements.

The real question is, why do we (as an industry) not use testing frameworks more to see if we could replicate those rare obscure bugs? If you can code the state, you now can reproduce it 100% of the time. The real answer seems to me, is that the industry isn't writing any or enough unit tests.

If your code can be unit tested, you can twist and turn it in many ways, if it's not an integration issue.

> Leave us be. We know what we’re doing.

No shade, this was my perspective until recently as well, but I disagree now.

The tipping point for me was the realisation that if I'm printing code out for debugging, I must be executing that code, and if I'm executing that code anyway, it's faster for me to click a debug point in an IDE than it is to type out a print statement.

Not only that, but the thing that I forgot to include in my log line doesn't require adding it in and re-spinning, I can just look it up when the debug point is hit.

I don't know why it took me so long to change the habit but one day it miraculously happened overnight.

Often you can also just use conditional breakpoints, which surprisingly few people know about (to be clear, it's still a breakpoint, but your application just auto continues if false. Is usually usable via right click on the area you're clicking on to set the breakpoint.

This post exactly.

I don't see any evidence that the 1% of bugs can be reduced so easily. A debugger is unsuitable just as often as print debugging is. There is no inherent edge it gives to the sort of reasoning demanded. It is just a flathead rather than a phillips. The only thing that distinguishes this sort of bug from the rest is pain.

When the print statements cause a change in asynchronous data hazards that leads to the issue disappearing, then what's the plan since you appear to "know it all" already? Perhaps you don't know as much as you profess, professor.

> Leave us be. We know what we’re doing.

No. You’re wrong.

I’ll give you an example a plain vanilla ass bug that I dealt with today.

Teammate was trying to use portaudio with ALDA on one of cloud Linux machines for CI tests. Portaudio was failing to initialize with an error that it failed to find the host api.

Why did it fail? Where did it look? What actual operation failed? Who the fuck knows! With a debugger this would take approximately 30 seconds to understand exactly why it failed. Without a debugger you need to spend a whole bunch of time figuring out how a random third party library works to figure out where the fuck to even put a printf.

Printf debugging is great if it’s within systems you already know inside and out. If you deal with code that isn’t yours then debugger is more then an order of magnitude faster and more efficient.

It’s super weird how proud people are to not use tools that would save them hundreds of hours per year. Really really weird.

Renderdoc!

So, uh, everything is important, and every engineer must know everything then?

I mean, don't get me wrong, I do agree engineers should at least be aware of the existence of debuggers & profilers and what problems they can solve. It's just that not all the stuff you've said belongs in the "must know" category.

I don't think you'll need valgrind or query planning in web frontend tasks. Knowing them won't hurt though.

I'd investigate why it won't run with debug info in the first place. That feels like the core problem here, because it prevents you from using some debug tools.

Of course that may require digging down pretty low, which is difficult in itself.

Edit: also there's split-debuginfo which puts debug info in separate file. It could help if the reason you can't run it is the debug info itself. Which feels unlikely, but :shrug:.

You can add debug info to release builds. In Cargo.toml:

    [profile.release]
    debug = true

claude code cli

Watchpoints (breakpoint that triggers when a given memory address is read/written/both) are the other huge UX advantage over prints. They're not quite as universal: you can almost always "print" by blinking an LED even if you don't have the luxury of a UART or other console, and some extremely low-end MCUs don't have any sort of watchpoints, but almost everything else does. A watchpoint that logs which source line wrote to the address in question & continues execution automatically is one of the easiest ways to debug memory corruption.

Looklike you invented "tracing", but since you added a hook at every "curly bracket", it would be much more detail than average tracing.

And slower of course, they are not free.

Looks like you invented telemetry

I read it as dealing with applications you only get shell access to and can't forward ports.

https://lemire.me/blog/2016/06/21/i-do-not-use-a-debugger/

A bit of counterpoint here

Interesting POV. I see it exactly the opposite: using a debugger most of the time feels like trying to see the current state of things without understanding what set of inputs led to it. Print debugging feels more like trying to understand the actual program logic that got us to this point, based on a few choice clues.

I’m not saying you’re wrong or I’m right, just that we have diametric opposite opinions on this.

Call stacks and reading code give very different views of the codebase. The debugger tells you what's happening, reading tells you what can happen in many situations at once. You can generalize or focus, respectively, but their strengths and weaknesses remain.

Readable code, though, is written with the reading view in mind.

set -x

I'm pretty sure in that interview at some point he realized becasue the debugger experience for developers using Linux sucks compared to Windows where he does most of his work.

Alot of programmers work in a Linux environment.

It seems like windows, ide and languages are all pretty nicely integrated together?

Assuming you meant C's printf, why would you subject yourself to the pain of recompilation every time you need to look at a different part of code? Isn't the debugger easier than adding printf and then recompiling?

Do you use snippets or something to help speed this up? Manually typing `printf("longvarname=%s secondvarname=%d\n", longvarname, secondvarname);` adds up over a debugging session, compared to a graphical debugger setup with well-chosen breakpoints, watches etc.

This is a solid answer.

I think it would be interesting to view this from a different angle. Perhaps "Lots of people who know of debuggers still use printf debugging, maybe they're not all wrong and there are advantages that aren't so clear."

Good print statements can become future logging entries for when software ships and debugging statements need to be turned on without source code access.

“All these senior/staff FAANG folks are using a different tool than the one I regard as essential.”

There are a couple of ways to resolve this conundrum, and you seem to be locked on the less likely one.

What if… that weren’t an essential skill?

I'm so used to bouncing between environments my code's running in (and which project I'm working on) that I tend to just assume I don't have debugger access, or at least don't have it configured for that environment, even when I do. Like I'm just in the habit of not reaching for it because so often it's not actually there. It rarely matters much anyway (though when it does, yeah, it really does).

Imagine I posted the bell curve meme with "print debugging" on both ends.

I'm surprised that you can get that far without seeing value in print debugging.