Read/write sequences to RAM are inconclusive regarding semantic equivalence. You can have identical sequences and semantics can still be wrong (e.g. when working with the FPU and the float stack, or instructions operating purely on registers). You can also have diverging sequences when having identical semantics, e.g. variables occupying different stack slots, despite computing the same thing.

Also, it feels like just fuzzing the function is slow, compared to simply checking bytes. Fast feedback allows agents to iterate much faster. At some point I wrote a tool that simply proves semantic equivalence, while ignoring irrelevant changes, e.g. register selection or stack slot changes. Turns out it was way too slow as a harness. I still used it to verify some of the remaining functions.

It turns out that getting functions byte-matching is actually not that complicated. Most functions are small and can be one shotted, and over time, agents figured out rules how to control code generation for more complex functions. They wrote down techniques and tricks on how to control register selection for example, or how certain control flow operations may impact basic block ordering, etc. Super insteresting what they figured out. So the further we got into the project, the easier it was to get functions exact.

Functional equivalence here, of course, depends on the completeness of the test suite, where byte-identical compiled artifacts does not.

(For example, your approach would not necessarily catch all the same overflow behaviors; the OP expressly claimed that "replicating all bugs" was also important, and many bugs are caused by certain overflow behaviors)

Only an LLM can give reasonably meaningful names to most functions and variables automatically, though.

Could you talk more about how you’ve used this technique? I want to start on a similar kind of AI-driven decomp project and I’m looking for any kind of edge I can get.

So you restricted it to implementing the same function (same inputs,outputs, dependencies as original?) and prevented the agents from making design decisions by keeping it's scope restricted?

Identical output isn’t ridiculous. It’s pretty much a requirement to ensure the game actually is the same in every way. These decomp projects are pitched as a high performance alternative to emulation.

No one is going to use it if it’s a kind of close but not really reimplementation.

Testing functional equivalence is also pretty much impossible. How would you for example test the new one has exactly the same bugs which haven’t been discovered yet. Or doesn’t introduce new ones? This stuff matters for speed runners.

I think you're overlooking an important point here. While the token consumption might feel extremely high, I think it's a bit deceiving.

Due to this strict harness, we were able to use extremely dumb models efficiently. I think at least 200-300B of tokens (maybe even more) were processed by Luna.

Given that Sol is 100x more expensive per token than Luna, a high token consumption doesn't necessarily mean high cost. So these 300B Luna tokens would equate to 3B Sol tokens.

Unfortunately a huge part of the logs is lost, so I can't do a detailed analysis on which models were used, how much they cost, etc.

Identical assembly output is the only way to get functional equivalence, otherwise you're getting equivalence in some percentage of situations, but not 100%. Your bug is my feature etc.

Let us note without irony that Corporate America is indeed here to ruin your fun. It is their call of duty to wage this type of modern warfare.

A C&D-130 in the air, heading for you.

Friendly reminder that corporate America came at Aaron Swartz.

The best models have O(1k tokens per second). A billion tokens, sequentially, takes 1-2 weeks. 500B takes 500x that ... not fast. The problem might not have required 500B tokens, but if it needed anything within a couple orders of magnitude then something like the given approach (or anything else yielding equivalent results in exchange for parallelism) was mandatory.

I wonder about this too. Although, cracking was always possible. So what changed is that people can now modify, improve proprietary software with zero technical knowledge. I can now tell Astra or Fable (maybe not Fable because of safeguards?) to add a feature to Adobe Photoshop and it might actually succeed even when prompting purely on a functional level (e.g. the functionality I want out of the program). It would take days, but I think it would succeed. Call it PhotoBench?

Non-SaaS, close sourced software is already dead, no? I think pretty soon we'll all be using bespoke software written by the AIs. I already use throwaway software designed for each job because it's so cheap to do so.

A future of all SaaS only works if you can't just tell a LLM what you want and get a custom implementation. I've always done a lot of personal projects, but my velocity has increased dramatically and I've replaced a number of projects that I used to pay for with custom ones that work well enough. I can't imagine that SaaS is going to be a viable model unless it involves a community that wants a unified experience, like a multiplayer game.

Most software organizations pay for is effectively a SaaS or something where software is a minor part of the artifact and support is where the real money is.

Well, I received a few DMCAs and thus stopped. We would've likely brought this as close to 100% as we can, but it's just pointless given the circumstances. So our belief and a few final tests were enough.

Fully agree.

Since even Fable 5.1 cannot reliably remember what it wrote into "memory" files before the last compaction, why would we trust it to not forget about some side-effect when handling assembly that is so verbose it'll surely exceed the context and, hence, necessitate compaction.

Call of Duty: Modern Warfare 2 (2009) (from a previous version of the page)

>what legal leverage

PIF owns EA, when they invite you to a meeting you might start to worry about foil lined room and carpentry tools lying around

> So, if you reverse-engineer game X and post reverse-engineered code, what exactly do you infringe, how and in which jurisdiction? What changes if it is done via LLM?

In 1986 there was a federal court case, Whelan Associates Inc. v. Jaslow Dental Laboratory, in which it was ruled that the "structure, sequence, and organization" of a computer program was protected by copyright, and thus independently produced software could be found infringing if it copied these elements, even if the code were not copied (or mechanically translated) verbatim. This led to a six-year period in which computer software enjoyed generous copyright protection, such that "clones" of copyrighted software were effectively infringing. It wouldn't be until the early nineties that other court rulings would tighten the rules again, notably Computer Associates International, Inc. v. Altai Inc.. The 3-step "abstraction-filtration-comparison" test has been used by most courts since 1992 to determine whether nonliteral parts of program code are eligible for copyright protection, and whether another, independently written program is infringing.

HOWEVER, the Whelan standard was never actually overturned or stricken from U.S. law due to legislation or litigation! And companies have sued and won under the Whelan standard! Most notably, Oracle in their copyright and patent case against Google regarding Java APIs in Android. Google ultimately prevailed, but only because the Supreme Court ruled that Google's use of the APIs was fair use; they did not overturn the Federal Circuit's finding that the "structure, sequence, and organization" of the Java declaration code was ineligible for copyright! And I doubt that the Supreme Court would similarly smile on a reverse-engineered complete video game!

I believe that these reverse-engineered projects infringe copyright, under the Whelan standard and perhaps under the stricter Altai standard as well. You do not get a free pass because the original code was in C and yours is in Rust; or because you created a slightly different version of each function in the original code.

It's not. Those 80% are calculated by a simple script that just compares bytes.

The obvious lesson is that you need some kind of test/harness, even for normal development. Your spec will always leave room for interpretation in a sense that is undesired.

Obviously matching functions byte-byte is not it, but I think with enough creativity, you can have guardrails and oracles almost everywhere.

E.g. before diving into feature development, you can have agents generate a test suite based on your implementation first. Then, when developing features, this test suite shows what broke and agents can double check if that was intended.

How that test suite looks like obviously highly depends on the environment.

It's not really comparable. When decompiling you only care about a one time result. Code quality doesn't matter much. Stability doesn't matter much. Only whether the result is accurate. Once you have the matching decompiled code, all tools used until that point will be usually thrown away.

Normal development isn't so lucky, you do care about code quality. So you have to review the code the agents write and handle the testing yourself. The agents will not do what you want, they'll do what they think you want, and you're the only person who knows what you think.

This stuff is all done on highly subsidized subscription plans. I suspect Antropic is quite happy to sell these people $100k of compute for $4k because it boosts their growth numbers and they can tell investors once they stop subsidizing, this will grow to $100k. Despite the fact that most of this stuff simply wouldn't be done without the subsidization.

Where are you getting 150-400 per million input?

https://developers.openai.com/api/docs/pricing https://platform.claude.com/docs/en/about-claude/pricing

OpenAI and Anthropic are both 10/mil in.

https://openrouter.ai/z-ai/glm-5.3#providers https://openrouter.ai/moonshotai/kimi-k3#providers Other frontier models are like 1-3/mil in.

Also, 500 billion is 500,000 millions. At the lower end of your 140/mil estimate that's 70 million dollars. Even at my 2/mil lookup for Chinese frontier models that's one million. Show your math for 100k-750k, please.

I struggle to believe that someone would find it worth that much money to have a decompiled version of a game they also commit to keeping private.

The vast majority of these tokens are cache input tokens so even at API price the cost would be much lower.