"Wouldn't a working approach be to just create a really low resolution 3D world in the traditional "3D game world" sense to get the spatial consistency. Then this crude map with attributes is fed into frame generation to create the resulting world?"

That's Minecraft. It does work. And when you turn around, then turn back to where you were facing, you don't feel like you're in a fever dream, because the landscape hasn't completely shifted.

The tech is neat and I'm sure worth publishing. The rhetoric accompanying it is terribly overblown. Of course that's as likely to be their University PR department as the authors.

> Wouldn't a working approach be to just create a really low resolution 3D world in the traditional "3D game world" sense to get the spatial consistency. Then this crude map with attributes is fed into frame generation to create the resulting world?

If you go "ultra-low" resolution, that's basically Minecraft or Luanti (formerly Minetest). There some mods for Luanti that let one generate a less "squary" world by adding slopes, but that's still heavily polygonal.

> It wouldn't be infinite, but on the other hand no one has a need for an infinite world either.

Minecraft has an infinite world (with glitches when you go very far from origin due to floating point errors). Luanti's world is finite (around 64000 m^3 because 16 bits coordinates; players move at 2-4m/s usually), some people are working to make it infinite. In my opinion, you are right that nobody needs an infinite world; the argument for an infinite world is that a finite world can be a problem on popular servers but in my eyes servers where this is a problem don't use enough the available space (in particular the huge vertical space that can be used to create multiple worlds) or shoot themselves in the foot by providing players with ways to move fast (mainly transports); increasing the player's speed actually shrinks the world.

I think there is a conundrum from the players and gameplay perspective. On one hand, you want a lot of space, but on the other hand you want to be close to other players (for commerce, play together, etc.). It's not enough to have an infinite world, you also have to have the gameplay that goes with it.

Another thing to consider are the interaction of mobiles (NPCs, enemies) with the terrain. Because nobody likes an infinitely empty world either. I haven't played an AAA game recently, but a decade ago even with precomputed terrain, some mobiles would look silly sometimes with steep terrains.

Not an expert, but I think procedurally generated terrain is generally fractal in nature, and is reproducible in that sense from a seed that is used in the generation. It is therefore recursive, as fractals are recursive.

A traditional neural network is a universal function approximator, however it is not recursive in nature, unless it is some sort of RNN. The transformer architecture, which this seems fairly similar to this one, is also not recursive in nature; although I believe, limited recursion can come about through CoT.

Therefore, I don't believe this could match the reproducibility, in an infinite sense, of a traditional procedural generator.

It's basically a dream. Maybe that's what dreams are? World prediction engine running with no sensor input and memory in read only mode?

What you really want is a story telling engine informing the world model and the humans need to generate power.

I think fever dream AI stuff like this would be nightmarishly terrifying in VR lol.

Still, would be super cool to try

I think we need devices that can produce gaussian splat videos to become as common as 2d cameras.

I think Ubisoft are leaders in that space. I saw algorithmic generation of New York style geometry for some behind the scenes video for The Division 2.

I wouldn’t be surprised if epic has similar.

There's so many researchers working in AI now that we can afford to explore all avenues. This probably isn't going to lead to a fully generative AI game engine, but I bet there'll be useful learnings along the way.

Generating coherent objects in an infinite world is probably harder.

Because you wouldn't have to wait for a human to write the engine and write the game

Click to play [...the video]

People have been doing that with Minecraft for over a decade. In the old days, once you got far away enough, the terrain generation would go haywire. Lots of videos from that time period of people exploring the "edge of the world".

Personally, these were the kind of glitches which made games feel magical and "real" to me as a kid. Being able to analyze a system by breaking it made it seem so much more tangible, like an actual place I had an NTSC-sized porthole into.

That community already exists because the current version of these types of AI game engines are constantly running into a surreal edge condition since they don't track things consistently when they go off frame.

Sigh, agreed!

Why is that sad?

I don't know if you're actually new here, or you've been reading HN for years, but your account is only 11 months old.

We talk about a lot of things here, but when we do talk about AI, we tend to prefer talk about things with code or papers.

This particular project has a paper. They're expecting to publish their code soon.

Here's the paper:

https://thematrix1999.github.io/article/the_matrix.pdf

You don't have to read it, but you may want to consider it if you want to learn something and/or contribute something meaningful to the conversation.

But then there would be no articles!

Why? It's a mostly democratic news aggregator site without much editorial overview, if you don't like it, downvote / don't upvote it, or write a client that filters topics you don't like out based on some keywords. You didn't need to open this page and comment on it if you don't like it, nobody's making you read things you don't like.

Check lobsters

Yes, and that someone should be you. Be the change you want to be.