> There is a long-lived and incredible community of classic gaming emulation work online for it to reference. And that community has created…conformance tests! Opus found them and built against one, that has >3k golden instructions

I'm doing something similar but not for an emulator, using tests other projects in a similar space use to validate things.

I also know a friend that feeds Claude Gherkin formatted test criteria, and apparently that yields insanely good Claude output.

Truly amazing. Quite thought provoking how it was helped by having high quality oracles. Eg. Qwenputer & good NES reference materials.

I guess humans need their oracles too: teachers, encyclopaedias, degrees, phds.

Bravo, awesome project, really creative!

You might like this: https://research.google/pubs/latent-reasoning-with-supervise...

I wonder if you could train an LLM while holding the weights implementing fixed-function regions constant, so the training would produce a model that natively knows how to use these tools inside it's own reasoning steps.

I suppose it doesn't need to necessarily be in the weights, a hybrid LLM + internal fixed function blocks that aren't encoded in the weights could be quite interesting.

Combine this with latent reasoning and it gets pretty wild.

A very useful fixed function region in my mind is a symbolic execution engine, capable of only following a well-defined grammar. Perhaps a lisp would be good too?

But wouldn't that be horribly inefficient? Instead of executing code by transforming it into a set of weights and running inference on them, wouldn't it be easier to just execute code as... code?

Totally. It's like how we put brainfuck interpreters everywhere.

You can just do that anyways.

The day is coming when we have widespread local inference on super powerful and efficient hardware and people will ask it what time the store closes, get distracted, and then ask it again instead of scrolling up.

Once our compute base shifts from silicon to human brains grown in vats, you'll be thankful you can run all your old deterministic compute tasks on flesh neurons!

Two years ago: nice toy.

18 months ago: it can do some things but it’s generally not useful.

12 months ago: wow; this can write ok code that a human has to do a lot of review to make sure it’s ok.

6 months ago: holy fuck I’m shipping great stuff with only the occasional redo.

3 months ago: Jesus Christ this is actually pretty fucking great. I have to work entirely differently; becoming a manager of my single agent launching sub agents.

1 month ago: my god I am shipping more, faster, and more useful than ever.

Today: we have fleets of highly effectively domain expert agents working entirely autonomously and shifting between fleets depending on domain, available usage remaining, and we’re shipping 10x with 100% better reviews than before—expanding by the day and self-improving its own harness.

—-

Next week: the agents will prioritize code golf.

The week after that: everything is written in ASM.

We’ll come full circle in 3y

e: fyi; this is my personal progression with LLMs.

This is a bit different though. It’s using transformer architecture to emulate almost at the circuit level. It’s probably even more inefficient than something that was trained to emulate a cpu, but with the advantage of actually being an exact emulation rather than learned approximation.

It’s a bit like when a human brain remembers facts, if you sat down and computed the hash values of the thing you were trying to remember, and traced out a DB lookup in your mind, it’d be a lot slower than just “oh yeah I remember that”.

> A while back I saw some people saying we would just have LMMs hallucinate an entire operating system every single frame. I guess technically it's now proven to be technically possible, but it would be thousands of times slower while requiring thousands of times more compute (a multi-million x difference from an ordinary OS!).

Gosh. That would be a very silly use of an LLM.

Bad use of any AI, even image/video models which are certainly less wrong than this. Stuff like DLSS show we're getting stuff that could generate a game's graphics frame-by-frame, but even though it's possible to have an entire game environment (not just graphics, everything) done by an AI, such things demo better than they play due to temporal coherency.

That said, having an AI make the code for each thing as you need it, might work somewhat OK eventually. Wouldn't trust it today (security, data integrity, and a tendency to cheat on tests), but eventually it could happen.

These systems in weights are proof of concepts. Surely there is room for optimization in there somewhere.

This is exactly like if someone implemented a CPU using nothing but CSS animations. Maybe a fun way to teach about turing completeness, but pretty meaningless otherwise.

Let's say you are developing a product, choose 1:

1) Have a single person vibe together a computationally inefficient solution in 1 month that runs transformers on some backend.

2) Hire a team to work for 2 years to design, perfect, optimize, plan, revise, and architect a solution.

The risk/reward for 1) is extremely better than 2). Given the turnover for entertainment in this day and age it doesn't make sense to do all this complex stuff.

Isn't that the same with e.g. human brain? Computer program can compute billions of equations, whilst human will struggle with just one.

They've already forced Electron on us, I'm sure they'll make LLM-approach just good enough too.

A middle ground would be an LLM taking structured inputs ("mouse clicked at X,Y") and making "draw" calls at whatever level: pixels, SVG, HTML/CSS. While the LLM's output tokens could be mapped to just these calls, it could also just do a bunch of tool calls that write to these formats as well -- it's a matter of efficiency.

We have Jev to prove LLMs that output constrained decisions are useful. Why not make LLMs just output UI natively?

At a fast enough inference speed, you could just connect the network to it.

So an LLM would then just take the network, user input, and output a UI, with tool calls to deal with memory and other resources. In theory, you don't need an OS any more.

> show the massive compute requirement between LLMs and ordinary programs completely agree, but it is at least raising the point that 150-300x is only a bit wider than things like framework/language choice... which is quite remarkable considering how much more open-ended running everything through an LLM is.

> hallucinate an entire operating system every single frame 1. I don't know that "hallucinate" is the right word here... any more than Windows 11 itself is a hallucination, for example 2. I don't personally think that this system is a million (thousands x thousands, from your attestation) times slower, but given Moore's Law, even that isn't the worst thing ever. this is the low water mark.

> have LMMs hallucinate an entire operating system (...) technically it's now proven to be technically possible

You're comparing apples to oranges there!

To hallucinate is arbitrary, you can use a 2B image param model, it will hallucinate an OS no problem* running real time!

What this post is about is completely different, which is actually emulating the complete system in an LLM - that is hundreds (or more) times slower.

*absulutely crappy quality tho, unusable OS in practice

It would be a natural extension of the current work in software. Every layer of abstraction/web/scripting language adds seemingly 10x in both ram and processing time, undoing the advancement of 1000x faster processors and ram increases.

But at some point we’re hitting a wall here. If we want our software to undo 100,000x more processing powers, we need that llm software emulation layer.

That's the case right now--but what might a customized model tailored only for a narrow set of OS functions behave like? I suspect its achievable with a lot less compute.

Now is that a desirable product? Would it succeed on the merits in the marketplace? Maybe it would...

This has the nice side benefit of letting the 1gb of weights compress down to ~3mb (since it’s mostly empty)

Sometimes projects have no point other than to show what can be done – and the pure thrill of doing is an end in itself … but I agree with you in the "so what" sense here … I feel like with this project there is meant to be a point (other than play for the sake of play and art for the sake of art and coding for the sake of coding) but I'm not seeing it either?

Apparently it's news to some people. It also helps combat the "LLMs ArE nOt DeTeRmInIsTiC" complaints.

Because it's fun

This work, and the original qwenputer work it builds on, are both technically uninteresting slop. HN eats it up though.

Not a climber then?

I imagine it would prove really difficult to keep generating valid instructions.

Doesn't work for all cartridges (I didn't run Opus that long, but it could keep going). Works with Super Mario.

But to answer your question, it's in the readme: https://github.com/dbreunig/nesformer#how-it-was-checked

Do you test every possible input/output combination on any CPU?

No, because that would be impossible.

You perform a series of more specific tests. In this case, it's "working" in so far as that Super Mario Bros. appears to be playable, for example.

Does not mean that it behaves perfectly according to some spec, or identical to another CPU (like the NES's 6502), of course, but it's clearly a working CPU to some 6502-compatible extent.

Yes, calling a Nintendo Entertainment System (NES) a "Nintendo" may have made sense in 1989 or 1990, but not in 2026.

The title is very "grandma who doesn't know the tech" coded. Every game system is the Nintendo, controllers are remotes, etc.