Oddly enough I can’t access that site, it just heats up my phone solving hashes. Gave up after about a minute and anubis had only made it less than halfway through.

Finding the bugs with LLMs is easy. Reviewing the output, cleaning it up, and making sure it doesn't break something else is the hard part.

In my experience, there are two ways to use AI: speed or quality. Speed is where you give the AI a task to do and you review it; quality is where you write the code yourself and you get AI to review it. Both are valid for different situations.

> It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal.

No big deal? It’s not like it’s free… tokens cost money.

I don't care if you call it an over-engineered looping machine or what, there are concrete benefits to using LLMs for this. They work faster than developing your own looping algorithm and more often produce useful results than not.

I dislike AI, but if AI finds real bugs then this is in my opinion objectively a positive thing. Of course the question is what constitutes a real bug.

But what's your expectation of the net?

Not custom. It's an existing module for a format called VPK. It's a quite trivial bug though, not exploitable apart from DOS and won't ever happen in a real file.

Not that it doesn’t have issues, but I’m not sure why you’d choose tmux of all things. It runs as a user and has no privileges to escalate. It was written for and is part of OpenBSD and follows their security hardening practices.

(There actually was one privilege escalation bug in tmux, but it actually seems like a distro packaging error. The distro setgid the executable so the resulting shell inherited the additional group. This didn’t require any exploit, that’s just how child process inheritance works.)

I don’t think tmux is the most worthwhile target because you’d need the user to either execute code locally (thus negating any point in targeting tmux) or rely on the user curl or cat some compromised document (in which case you’re better off targeting curl or cat).

the fact that you can vibe code a harness to look for bugs in a famously complicated C codebase is intern level stuff and hardly news

It seems like this would have been pure fantasy not that long ago though. So why isn’t it noteworthy again? I don’t really follow what you’re complaining about.

> but can't you just mark all "/" as potential divide by zero errors?

If you’re accepting large false positives rates: yes.

If you want users to take your warnings serious: no.

(Nitpick: you certainly don’t want to flag _all_ of them. Divisions by non-zero constants definitely should be excluded, for example (integer division by -1 can lead to overflow, but that would be a different warning))

Fuzzers find inputs, not just "potential" errors that aren't triggerable.

If it's possible for program execution with some particular input to lead to a divide-by-zero, that's a bug, especially if the program is expected to be able to handle malformed inputs, or perhaps even deliberately malicious ones. It's not trivial to determine whether a program does this correctly. If it was, program analysis would be easy.

Division can 'go wrong' for certain inputs, but it's not just division. In C, signed integer addition, subtraction, and multiplication, all give undefined behaviour on overflow.

As 'Someone' already pointed out, it's not helpful to just flag all uses of the division operator, or of other potentially dangerous operators. Minimising false positives is one of the core challenges of program analysis.

What are you suggesting and how would it be different than how SIGFPE already works?

I mean there could be a guard clause? But yeah, seems like this could be statically evaluated like how some IDEs see a null check and don’t complain about nullability within the same scope.

Lots of projects run their own git or forgejo or similar. I run my own private forge, and it has a higher uptime than GitHub. (A shockingly low bar, tbh)

It’s surprisingly simple to setup, and the hardware requirements are pretty small for a private or small forge, as it’s usually a relatively small number of users/repos/etc.

You can add several remotes to your git, and I'd recommend you do so.

> It is interesting that FFmpeg has its own Git server. Maybe we should move there too?

Git is a DVCS. I know many people only ever used Git through Github and forgot what the 'D' in DVCS means but whether or not they remember what the 'D' stands for, running your own Git server is trivial. Especially in this day and age of LLMs were you can just ask: "Clone this repo and convert it to base Git repo and serve it on the LAN PLZ KTHX".

The result is going to be more stable than Github and, arguably, more secure too.

it's widely used but in "industry" applications. so ffmpeg is probably being used in a lot of offices (studios) and maybe even being included in end user software.

Every day, we stray closer to Haskell. Dare I say it: good!

What we need are refinement types, where there’s a base type and a predicate. F* has this:

     val (/) : int -> (divisor:int { divisor <> 0 }) -> int

The only way to achieve this is to either put a runtime software check on a variable whenever it's assigned/used, or to literally add hardware support in processors themselves which literally throws an interrupt when a "neverShallBeZero" variable is assigned to zero.

There's no viable way to statically prove at compile-time that these variables will never become zero at runtime, ultimately forcing a system of endless runtime checks (be it software or hardware)... which is why processors already throw exception interrupts when division by zero is attempted.

And also cannot be INT_MIN, otherwise -1 / INT_MIN is undefined behaviour(!) in C and C++.

It would be more flexible for a compiler to reuse the range analysis logic used in optimizations for statically verifiable divide by zeros. That way you could extend it to other things like statically verifiable overflows.

For stuff like niche value optimization sure. For practical arithmetic code, nah. Like with this bug, all that changed is that garbage data in gives the user an error that they tried to process garbage data. Adding a new type doesn't make the code better, it just moves the error around. And you really don't want an infix division operator to fail to type check if the right hand side isn't a nonzero type, do you?

...they did.

Those sneaky LLMs going 7 years into the past and committing as a human:

https://code.ffmpeg.org/FFmpeg/FFmpeg/commit/8eda3c7f91e1a5b...

You're not reading it right. The bug was found using a vibecoded fuzzer.