Author here. I agree with you - the number of metrics I can experiment with in Pong is limited. Chess and Go are next for me.

Overall, the simplicity of this project has helped me test the waters before diving into more complex territories. The underlying pipeline isn't bad - the approach of collecting events, periodically generating metrics from them, prioritizing them, generating commentary text, queuing those outputs, and then synthesizing speech should serve as the core for similar work.

It's also given me some intuition on how I can construct an "ecosystem" of data surrounding live action, to add a layer of realism to the narratives.

>Could you apply the same idea to chess or Texas Hold 'Em? I feel like the additional complexity of those games could lead to more interesting commentary.

The additional complexity in something like hold 'em lends itself extremely well to LLM generated commentary.

i think pong being so simple is why this is funny and interesting

Naw there are tricks you can use to pipeline these things so that apparent latency is under 500ms even with significant game state history awareness, and also to interrupt ongoing but freshly out of date commentary.

I couldn’t get it under 250ms though (for rocket league), but the tech should be better now than 2024.

Author here. TTS and script generation can be a bit of an overhead for now, which is why I've worked with metric aggregates - 30+ bounces rather than exactly 33, for example. For this game, one might ideally want this overhead to be less than the time it takes for the ball to bounce from one paddle to another, which can be around 1–2 seconds. However, there may be another strategy to (maybe?) overcome this: start synthesizing numbers (ignoring the fractional part) using TTS and cache them for both commentators. Then, patch those audio clips together after core part is synthesized. It should be doable, I think - I just haven't gotten to it yet. Note that matching the excitement and tempo of core commentary with those numbers is key - otherwise, it will feel janky.

It's funny that even in a simple setting the AI couldn't avoid to hallucinate. As this clearly isn't a champions match.

[dead]

It raaaaaaannnnnn, no exceptiooooonss!!! Can you believe it??? Can you believe it how he compiled that code? What a beauty, what a beautiful job he's done...

I wonder if NotebookLM's podcast function could be used for this, to comment on code with the spirit of a Latin American soccer commentator. Because having it comment code is already pretty useful if you don't want to explain others what you have been doing. It can do that pretty well for you.

Someone could totally make this as a vscode (etc.) extension

Like this https://www.youtube.com/watch?v=zgq236m0bcQ

Or an AI doing the part of the pair programmer who doesn’t have the keyboard.

You might not know this if you don't actually play these games (Madden, 2K for NBA, MLB The Show), but the commentary is extremely high quality, sometimes comparable to the TV broadcast with riffs and tangents as well as describing the action. Over many years of producing these games they have continually refined the process. Of course, eventually you will hear repeating dialogue if you play the games enough, but I think the baseline quality is going to be _very_ hard to replicate with an LLM.

Yeah I was thinking the same. No more "They've really got to want to win this. This is a game of two halves. Etc."

Though tbh I found it still pretty annoying. Maybe just the tone of voice though, and it's clearly not actually connected to what's happening in the game.

I imagine the major sports game players are working on this.

None that I can think of. The Finals has AI generated voiceovers for its announcers, but in that case the lines are pre-written and voice clips generated ahead-of-time so it just reeks of penny-pinching by cutting out real voice actors, rather than using the tech to do things that genuinely weren't possible before.

https://www.youtube.com/watch?v=kZ87wiHps9s

Have done an interactive commentator for rocket league that is also simultaneously your duo partner. Works quite well. This was in October 2024 so the tech is there and even better now.

E-Sports needs more commentators from Latin America or the Middle East.

If there is a programmed connection to the physics for the in-game commentary then it should be here: https://github.com/pncnmnp/xpong/blob/main/main.py#L212

https://github.com/pncnmnp/xpong/blob/main/main.py#L289:

  "- **Shot Angles:** Derive each shot's angle from the (vx, vy) vector:\n"
  "    • Steep angles (>45°) become daring corner lobs or sharp cross-courts.\n"
  "    • Moderate angles (15°-45°) look like graceful arcs that test court coverage.\n"
  "    • Shallow angles (<15°) play out as direct, flat drives down the line.\n"

Hah, now I'm thinking of an AI that knows your personality and steers you do things you've always been fearful or anxious about. "Steve sees a 10/10 girl, the kind he's always felt inadequate to talk to, but he's a different man now, and he's going to tell her 'Tickle your ass with a feather?' in 5... 4... 3... 2...".

Hah, my next startup is an AI-Assist Pick-Up Artist. But that's the "Lamborghini-desiring Crypto-Bro" package that's 49.95 USD/month, the entry level feature would encourage you to go to the gym and eat your vegetables.

AI voices talking to you... now the hallucinations are actually in your head!

It’s 2025. We’ve been able to render small circles for a while now!

Seriously though, the entire graphics display is much more hi res than the original, and it’s not trying to emulate the original resolution. So one slightly more serious way to answer the question is, all the graphics are higher resolution, it’s just that you notice it more when it comes to the ball.

> Swatch Whisperer

According to Google, you’re only the second person in recorded human history to use these two words together.

> Greg: And oh! There it is! That final middle patch—she’s going matte, folks. This wall is becoming a canvas of completion, a poetic stillness in a chaotic world. I feel like I just watched Cinderella get her slipper and a Roth IRA.

> Marsha (tearfully): This… is why I do this job. For moments like this. For the shimmerless silence. For the slow, glorious commitment to finality.

> Greg: And so we leave you, dear viewers, staring into a flat, fully-dry future. The room has changed… and so have we.

I’m getting major Broomshakalaka vibes in the best possible way.

https://www.youtube.com/watch?v=zt2uIhAvQZ8