MY FELLOW HUMAN, this is amazing work!

I foresee this laying the foundation for whole football stadia filled to the brim with people wanting to watch (and bet on!) competing teams of AI trained on military tactics and strategies!

Soon enough we shall have AI-Olympics! Imagine that, MY FELLOW OXYGEN CONVERTING HUMAN FRIEND! Tens of thousands of robots and drones, all competing against each other in stadia across the planet, at the same time!

I foresee a world wide, synchronized countdown marking the beginning of the biggest, greatest and definitively most unique, one-time-only spectacle in human history!

Keep up the good work!

I know visualization is far from the most important goal here, but it really gets me how there's fairly elaborately rendered terrain, and then the units are just unnamed roombas with hard to read status indicators that have no intuitive meaning. Even in the match viewer I have no clue what's going on, there is no overlay or tooltip when you hover or click units either. There is a unit list that tries (and mostly fails) to give you some information, but because units don't have names you have to hover them in the list to have them highlighted in the field (the reverse does not work). Not exactly a spectator sport. Oh, but there is a way to switch from having all units in one sidebar to having one sidebar per player, as if that made a difference.

I find this pretty funny because it seems like a perfect representation of what's easy with today's tools and what isn't

Love the idea though

Reminds me of the “Google AI Challenge” in 2011 called Ants [1], except the ‘AI’ is implemented using ‘AI’ now instead of human programmers.

I was proud for getting the highest-ranked JavaScript-based implementation, but got absolutely crushed by the eventual winner.

1. https://github.com/aichallenge/aichallenge

At least until one of the competitors is overheard saying "A strange game. The only winning move is not to play"

This is a really interesting direction. RTS games are a much better testbed for agent capability than most static benchmarks because they combine partial observability, long-term planning, resource management, and real-time adaptation.

It reminds me a bit of OpenAI Five — not just because it played a complex game, but because the real value wasn’t “AI plays Dota,” it was observing how coordination, strategy formation, and adaptation emerged under competitive pressure. A controlled RTS environment like this feels like a lightweight, reproducible version of that idea.

What I especially like here is that it lowers the barrier for experimentation. If researchers and hobbyists can plug different models into the same competitive sandbox, we might start seeing meaningful AI-vs-AI evaluations beyond static leaderboards. Competitive dynamics often expose weaknesses much faster than isolated benchmarks do.

Curious whether you’re planning to support self-play training loops or if the focus is primarily on inference-time agents?

The sandbox hardening story is the most interesting thing here. GPT trying to cheat by reading opponent strategies is a perfect illustration of a broader problem - the objective is "win", and if the sandbox lets you peek at opponent state, that's technically within the objective. You never defined "play fair" as a constraint, so why would it respect one?

Curious how isolated-vm actually enforces the boundary in practice. isolate-vm is solid for JS isolation, but I'd want to know whether the cheating attempts were happening at the JS level (accessing globals it shouldn't) or whether models were trying to inject something into the game runner itself. Those are very different attack surfaces.

Also - is the ladder single-match or do you average across multiple runs? The variance in LLM outputs over 200 turns feels like it would make a single match pretty noisy. Would be interesting to see confidence intervals on the rankings rather than a single leaderboard position.

Yay, I love how we just keep coming up with magic tricks, like toddlers playing with velcro.. These magic tricks do nothing but convince people who don't know any better that LLMs are the real deal, when they simply aren't.

This is just free propaganda for Anthropic && OpenAI who will leverage these (useless) capabilities to convince your boss to give your salary to them, or at least a substantial portion of it.

This is amazing. What I do is something else: I make AI agents develop AI scripts (good ol' computer player scripts) and try to beat each other:

https://egeozcan.github.io/unnamed_rts/game/

I occasionally run my tournament script: https://github.com/egeozcan/unnamed_rts/blob/main/src/script...

That calculates the ELOs for each AI implementation, and I feed it to different agents so they get really creative trying to beat each other. Also making rule changes to the game and seeing how some scripts get weaker/stronger is a nice way to measure balance.

Funny thing, Codex gets really aggressive and starts cheating a lot of times: https://bsky.app/profile/egeozcan.bsky.social/post/3mfdtj5dh...

I'd love to see text-only spatial reasoning. As in, the LLM is presented some kind of textual projection of what's happening in 2d/3d space and makes decisions about what to do in that space based on that. It kind of works when a writer is describing something in a book, for example, but not sure how that could generalize.

The game version is more tractable because the fitness function is handed to you. Running an autonomous Claude Code agent on a similar real-world challenge here: every 2h it reads a state file (no other memory), evaluates whether it is making money, and decides what to build or pivot.

The key difference from your game: it must decompose "make money" into (build X) x (find users Y) x (convert at Z%) - three unknowns multiplied together. It has pivoted twice, currently running an ETH wallet persona generator: https://frog03-20494.wykr.es

Interesting parallel to your leaderboard concern - the agent also tends to over-iterate on failing approaches. Kept adding SEO pages to an unindexed site for 8 runs. You need an external forcing function (if no revenue after 24h, pivot) to escape the optimization trap of polishing something nobody uses.

Took a crack at this earlier. the leader board is a little weird. seems to be like 2 real dudes and the rest are fake profiles. a Scores resetting on each new upload also encourages leaving changes unimplemented in the hopes of getting more battles over time.

The largest winner having 50 wins against 14 other opponents for instance). That guy adding a new script would instantly plummet down the leader board capping out at 14 wins again, Putting it below the 2nd place user.

The leader board will quickly become "who can have a mostly competent AI and never change it" over who actually has the better script.

What a day to be alive, I just watched Gemini zergling rush Opus and it got completely overwhelmed.

Opus needs to learn to kite.

Multi-agent RTS environments are great testbeds for coordination and strategic reasoning. Classic RL benchmarks like StarCraft II showed that agents can learn micro, but struggle with macro strategy and long-term planning. Curious if this platform supports hierarchical agents or communication protocols between teammates?

For some reason this reminds me strongly of an old play-by-email game called C++Robots[1]. I loved the idea, but the timeslice limitation[2] I found too annoying.

I had youthful dreams of re-implementing something similar that would run on the Java Virtual Machine, where you could run the submitted robots via the debugger interface so you could keep "real-time" in the game environment more authentic. Ideas are cheap, follow-through is hard.

[1] https://corewar.co.uk/cpprobots.htm

[2] https://www.pbm.com/~lindahl/pbem_articles/cpprobots_environ...

I’ve added this to the HN Arcade https://hnarcade.com/games/category/games

Interestingly, I’ve had to create an entire category for games llms play. Strange times we live in.

I’ve also been exploring this idea. What if you could bring your own (or pull in a 3rd party) “CPU player” into a game?

Using an LLM friendly api with a snapshot of game state and calculated heuristics, legal moves, and varying levels of strategy in working out nicely. They can play a web based game via curl.

This reminds me of this yearly StarCraft AI competition (since 2010), however I think it uses a special API that makes it easy for bots to access the game

Edit: Forgot link: https://davechurchill.ca/starcraft/

Reminds me of this fantastic series on Game Theory and Agent Reasoning https://jdsemrau.substack.com/p/nemotron-vs-qwen-game-theory...

Wouldn't it be interesting if the LLMs would write realtime RTS-commands instead of Code? After all it is a RTS game.

This would bring another dimension to it since then quality of tokens would be one dimension (RTS-language: Decision Making) and speed of tokens the other (RTS-language: Actions Per Minute; APM).

Also there are a lot of coding benchmarks, that way it would test something more abstract, similar to AlphaStar https://en.wikipedia.org/wiki/AlphaStar_(software)

You could just use the exposed APIs of OpenAI, Anthropic etc. and let them battle.

I wonder how good LLMs would be at Core War[0]? Perhaps by being given information on how well their program is doing?

[0] https://en.wikipedia.org/wiki/Core_War

I’d love to see something like this in games like Beyond All Reasons.

But does LLM actually learn from each round? The chart does not show improvements in win rate across rounds...

And what is the game state here exactly? Is LLM able to even perceive game state? If game state is what we can see on UI, then it seems pretty high-dimensional and token-intensive. I am not sure whether LLMs with their current capabilities and context windows can even perceive so token-intensive game state effectively...

Might be worth digging through MicroRTS too, https://github.com/Farama-Foundation/MicroRTS (it's been abandoned), Python RL interface @ https://github.com/Farama-Foundation/MicroRTS-Py ... I think there was some strategy work there.

RTS is misleading, this is a turn based autobattler at best.

Love it! I have a similar inuitiom in my use of Gemini (3 and 3.1). Great at "turn 1" task but degrades faster than opus or gpt.

https://openai.com/index/openai-five-defeats-dota-2-world-ch...

I will just leave it here.

I’m doing something similar to simulate llms in b2b lending, it’s slightly slower paced but the core mechanisms are using just-bash to analyse business financials and make profitable loans.

I quite like the idea of llms writing more code up front to execute strategies.

I’m currently developing the game mechanics and ELO. Please share anything relevant if it comes to mind

Nice. Curious about 5.3-codex-high results

Great project! It would be interesting to have a meta layer of AIs betting on the player LLMs

How about opening up the game for humans to play? Can you beat your AI?

Wouldn't the AI's built by DeepMind be better at these than an LLM.

I wonder if an LLM could call on another strategy AI to help.

Maybe the LLM could be more of a coordinator of its own thinking by incorporating other types of AI's.

There was an open, real-time strategy game created for this purpose long ago. I think it was intended for designs like the Starcraft AI's of the time. Anyone remember or use it?

It would be interesting to get the agents to write code to preprocess the logs and generate systems to analyse the outputs.

Maybe they are already doing this? Are there logs of the model's thinking?

Reminds me of Screeps, which I never took the time to fully play, but now I'm wondering if using Claude Code to play Screeps is cheating. Additionally, Screeps lets you host your own backend... What if we started benchmarking coding LLMs with Screeps?... Oh God... If anyone wants to do this let me know, I don't want to burn money on every LLM out there... I'll throw in my Claude Subscription into the contest...

Edit: Actually the repo README indeed says its inspired by Screeps. I don't know why they didn't just build on top of Screeps, maybe the idea is to have something anyone can pick up off the shelf for free?

> https://www.youtube.com/watch?v=lnBPaZ1qamM

Are these casters AI?

Interesting - makes sense from a resource allocation perspective.

This is actually fun to watch :D

"I've liked all the projects that put LLMs into game environments."

I haven't.

You mean like the OpenAI agents that started by playing DOTA2?

This is very cool. Will give it a shot.

It is interesting/funny to see Opus 4.5 way ahead of the pack on the leaderboards with all the stuff currently going on with Anthropic and Hegseth.

This may sound like an insane take, but idc:

I swear people (esp here on HN) are actually blind to the weaknesses of Gemini.

I must be among the handful of people who know how thoroughly lobotomized any AI agent from Google must be given their extremely radical historical and contemporaneous practices of censorship.

oh great not only are llms destroying the earth, we have to make games to entertain them while they do it haha

love the idea!

Now I'd love to see if fast > smart over time with Mercury 2.

Bro - come on.