I’m fascinated by the aesthetic of this technique. I remember early versions that were completely glitched out and presented 3d clouds of noise and fragments to traverse through. I’m curious if you have any thoughts about creatively ‘abusing’ this tech? Perhaps misaligning things somehow or using some wrong inputs.

I remember splatting being introduced as a way to capture real life scenes, but one of the links you have provided in this discusson seems to have used a traditional polygon mesh scene as training input for the splat model. How common is this and why would one do it that way over e.g. vertex shader effects that give the mesh a splatty aesthetic?

From the article:

>Evercoast deployed a 56 camera RGB-D array

Do you know which depth cameras they used?

Can such plugin be possible for Davinci Resolve, to have merge of scene captured from two iPhones with spatial data, into 3D scene? With M4 that shouldn’t be problem?

Great work! I’d love to see a proper BTS or case study.

Hi David, have you looked into alternatives to 3DGS like https://meshsplatting.github.io/ that promise better results and faster training?

Random question, since I see your username is green.

How did you find out this was posted here?

Also, great work!

I've been mesmerized by the visusals of Gaussian splatting for a while now, congratulations for your great work!

Do you have some benchmarks about what is the geometric precision of these reproductions?

nice work.

I can see that relighting is still a work in progress, as the virtual spot lights tends to look flat and fake. I understand that you are just making brighter splats that fall inside the spotlight cone and darker the ones behind lots of splats.

Do you know if there are plans for gaussian splats to capture unlit albedo, roughness and metalness? So we can relight in a more realistic manner?

Also, environment radiosity doesnt seem to translate to the splats, am I right?

Thanks

Really cool work!

[dead]

[flagged]

[flagged]

Take the money and never admit to selling this shit. Why would you ever willingly associate your name with this?

> so many artists reached out to have him do this style of mv, instead of wanting to collaborate on music

Well yes, the visuals are awesome, while the music… isn’t.

This is sooo sick. I’m a total hip hop unc and haven’t caught up for a decade and a half now and I think the music is great as well. Pairs perfectly with the visuals. Hope this guy makes it, a creative one of one.

His stuff is already on repeat. Thanks for the recc, love this site.

You're saying Nial used guassian splatting for his video? Or the style of camerawork, staging, and costuming is similar?

Put another way, is this a scientific comparison or an artistic comparison?

Hah, for the past day, I've been trying to somehow submit the Helicopter music video / album as a whole to HN. Glad someone figured out the angle was Gaussian.

I run a programming company and one of my sales people was surprised to see I liked soundcloud rap. I was like:

What did you expect?

>Classical music?

Nah I like hype, helps when things are slow.

And nearly a Carti post at the top of HN

Is he wearing... hair curlers?

I know right? I'm into both things tech and hip hop, didn't expect them to collide

yeah that had me do a double take lol

Why is that “cool” or desirable?

1. Create a point cloud from a scene (either via lidar, or via photogrammetry from multiple images)

2. Replace each point of the point cloud with a fuzzy ellipsoid, that has a bunch of parameters for its position + size + orientation + view-dependent color (via spherical harmonics up to some low order)

3. If you render these ellipsoids using a differentiable renderer, then you can subtract the resulting image from the ground truth (i.e. your original photos), and calculate the partial derivatives of the error with respect to each of the millions of ellipsoid parameters that you fed into the renderer.

4. Now you can run gradient descent using the differentiable renderer, which makes your fuzzy ellipsoids converge to something closely reproducing the ground truth images (from multiple angles).

5. Since the ellipsoids started at the 3D point cloud's positions, the 3D structure of the scene will likely be preserved during gradient descent, thus the resulting scene will support novel camera angles with plausible-looking results.

Gaussian splatting is a way to record 3-dimensional video. You capture a scene from many angles simultaneously and then combine all of those into a single representation. Ideally, that representation is good enough that you can then, post-production, simulate camera angles you didn't originally record.

For example, the camera orbits around the performers in this music video are difficult to imagine in real space. Even if you could pull it off using robotic motion control arms, it would require that the entire choreography is fixed in place before filming. This video clearly takes advantage of being able to direct whatever camera motion the artist wanted in the 3d virtual space of the final composed scene.

To do this, the representation needs to estimate the radiance field, i.e. the amount and color of light visible at every point in your 3d volume, viewed from every angle. It's not possible to do this at high resolution by breaking that space up into voxels, those scale badly, O(n^3). You could attempt to guess at some mesh geometry and paint textures on to it compatible with the camera views, but that's difficult to automate.

Gaussian splatting estimates these radiance fields by assuming that the radiance is build from millions of fuzzy, colored balls positioned, stretched, and rotated in space. These are the Gaussian splats.

Once you have that representation, constructing a novel camera angle is as simple as positioning and angling your virtual camera and then recording the colors and positions of all the splats that are visible.

It turns out that this approach is pretty amenable to techniques similar to modern deep learning. You basically train the positions/shapes/rotations of the splats via gradient descent. It's mostly been explored in research labs but lately production-oriented tools have been built for popular 3d motion graphics tools like Houdini, making it more available.

It’s a point cloud where each point is a semitransparent blob that can have a view dependent color: color changes depending on direction you look at them. Allowing to capture reflections, iridescence…

You generate the point clouds from multiple images of a scene or an object and some machine learning magic

This 2-minute video is a great intro to the topic https://www.youtube.com/watch?v=HVv_IQKlafQ

I think this tech has become "production-ready" recently due to a combination of research progress (the seminal paper was published in 2023 https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/) and improvements to differentiable programming libraries (e.g. PyTorch) and GPU hardware.

For the ELI5, Gaussian splatting represents the scene as millions of tiny, blurry colored blobs in 3D space and renders by quickly "splatting" them onto the screen, making it much faster than computing an image by querying a neural net model like radiance fields.

I'm not up on how things have changed recently

https://aras-p.info/blog/2023/09/05/Gaussian-Splatting-is-pr... and for a visual demo of the result https://antimatter15.com/splat/

This is a REALLY good video explaining it. https://www.youtube.com/watch?v=eekCQQYwlgA

I found this VFX breakdown of the recent Superman movie to have a great explanation of what it is and what it makes possible: https://youtu.be/eyAVWH61R8E?t=232

tl;dr eli5: Instead of capturing spots of color as they would appear to a camera, they capture spots of color and where they exist in the world. By combining multiple cameras doing this, you can make a 3D works from footage that you can then zoom a virtual camera round.

Awesome work, incredibly well done! What was the process like for setting the direction on use of these techniques with Rakim? Were you basically just trusted to make something great or did they have a lot of opinions on the technicalities?

Grin Machine knocked this out of the park!

Great job, Chris and crew!

Hi, I'm one of the creators of GSOPs for SideFX Houdini.

The gist is that Gaussian splats can replicate reality quite effectively with many 3D ellipsoids (stored as a type of point cloud). Houdini is software that excels at manipulating vast numbers of points, and renderers (such as Octane) can now leverage this type of data to integrate with traditional computer graphics primitives, lights, and techniques.

My bad! I am the author. Gaussian splatting allows you to take a series of normal 2D images or a video and reconstruct very lifelike 3D from it. It’s a type of radiance field, like NeRFs or voxel based methods like Plenoxels!

Corridor has done some great stuff with Gaussian Splats, I recommend this video for a primer!

https://youtube.com/watch?v=cetf0qTZ04Y

Reminds me of Kurtwood Smith’s piping sales pitch in The Patriot

Kinda, it all depends on platforms and eras. As a demoscener I see more of techdemo, indiedev and "game-mods" vibes in much of it (especially certain kinds of jankiness feel of visuals in some parts that demosceners doing more high end in general try to avoid but is more meme-worthy in game-mods/indiedev circles). Demos often aim for a bit more clean visuals (and palettes), blurriness is often tuned in another way.

Sadly, much of the demoscene is in a bit of a navel-grazing retro computing phase, many active ones today are "returners" from the C64 and Amiga eras whilst PC sceners of the 90s either dropped off for money, games or kids.

It's also the sheer work-effort, demoscene in the 90s and early 00s could focus on rendering while visual art pipelines didn't matter as much, as graphics cards got better it was obvious that the scene was falling behind the cutting edge games (both in asset due to workload and hacks required for graphics cards to render realistically).

The introduction and early popularization of SDF rendering turned the scene a bit more relevant again, but it's also been masking a certain lack of good artists since programmers could create nice renderings without needing assets.

However, to match something like this video in creativity would require a lot of asset workload (and non-trival rendering), and that combo is not really that common today sadly.

Funnily enough, I was actually discussing just Gaussian Splatting as a solution for more "asset heavy" demos about a year ago with another scener friend, but sadly there's a tad of a stigma culturally as NN/"AI" methods has been fairly controversial within the scene, aside from programmers there are both visual and music artists, and among those camps it's not really a popular thing.

It's still mostly a method though, and SDF rendering + GS could in the end be a saviour in disguise for the scene to go beyond just rendering and bring back a bit more story-telling to the scene.

The aesthetic here is at least partially an intentional choice to lean into the artifacts produced by Gaussian splatting, particularly dynamic (4DGS) splatting. There is temporal inconsistency when capturing performances like this, which are exacerbated by relighting.

That said, the technology is rapidly advancing and this type of volumetric capture is definitely sticking around.

The quality can also be really good, especially for static environments: https://www.linkedin.com/posts/christoph-schindelar-79515351....

Several of ASAP's video have a lo-fi retro vibe, or specific effects such as simulating stuff like a mpeg a/v corruption, check out A$AP Mob - Yamborghini High (https://www.youtube.com/watch?v=tt7gP_IW-1w)

Knowing what I know about the artist in this video this was probably more about the novelty of the technology and the creative freedom it offers rather than it is budget.

For me it felt more like higher detail version of Teardown, the voxel-based 3d demolition game. Sure it's splats and not voxels, but the camera and the lighting give this strong voxel game vibe.

Yes, they talk about this in the article and that’s exactly what they did.

I wonder if you are thinking Source engine? I was getting serious skibidi toilet vibes during several parts of this video.

We will be able to have imax level 3D technically today if you feed it the correct data

A very cool aesthetic and application of technology, thank you for sharing.

Loving this; great work! Do you talk about the process anywhere in more depth?

Cool aesthetic!

Very cool!

Good idea - we'll put that link in the toptext as well. Thanks!

Similarly, the music video for Taylor Swif[0] (another track by A$AP Rocky) is just as surrealistic and weird in the best way possible, but with an eastern european flavor of it (which is obviously intentional and makes sense, given the filming location and being very on-the-nose with the theme).

0. https://youtu.be/5URefVYaJrA

Watch the video to the very end: the final splat is not a gaussian one.

My wife said the same thing, but it gets better after the intro.

I loved the video. Didn't get the motion sickness myself.

Yes, just clicked on the video, instantly nauseous. I get motion sick generally.

Same. It's very cool but left me badly motion sick.

For sure!

All art is derivative. I don't care for this music, but being derivative isn't a real criticism.

It's a post about the visuals not the libretto.

If you're looking for bars sans pussy, read on: https://www.youtube.com/watch?v=yKifJ4Q5ph0

I prefer when artists make music they intrinsically want to make — not what others want them to make.

It seems the numerous leaks and trials took their toll.

I don’t disagree with you—I felt “Tailor Swif,” “DMB,” and “Both Eyes Closed” were all stronger than the tracks that made it onto this album.

But sometimes you’ve gotta ship the project in the state it’s in and move on with your life.

Maybe now he can move forward and start working on something new. And perhaps that project will be stronger.

Im sure it was more like, “hey babe, can I get a few millions to go in the studio and experiment/make some art?” And then she was like, “yeah go for it! Make some weird shit.”

If I was in his position I’d probably be doing the same. Why bother with another top hit that pleases the masses.

Because it was awesome. But also, leaks probably.

You might consider why this article which has nothing to do with AI as you know it (except for the machine learning aspects of Gaussian splatting), and was produced by a huge team of vfx professionals, has made you think about AI democratising culture (despite the fact that music videos and films have been cheap to make for decades). Don’t just look for opportunities to discuss your favourite talking points.

I really don’t see the connection. A$AP isn’t a noob

The whole rap/hip-hop scene got jump-started by 1977 NYC blackout electronics store lootings.

Say you have a photo, but you want to be able to explore it in 3d so you put it into fortnite but when you move you can't see behind objects because the photo never "saw" behind object.

So you decide to take lots and lots of photos at every single angle possible, but you need a way to link these all together, so you decide that each Centrepoint of the image is a "gaussian". These splat everywhere.

Now you have taken all of these photos and you can now explore the image in fortnite because you took thousands of images of every possible view!

But what if you didn't want to just look at the frozen image in a landscape in fortnite, instead you wanted to use a man dancing in your new upcoming YouTube video called helicopter.

If you isolate this person (let's say taking all these photos on a green screen) you now have a 3d like recording model, you can reshoot and "scene" on-top of something else (like a 3d diarama like in your video!)

Same, I found this video explainer here:

https://www.youtube.com/watch?v=Tnij_xHEnXc

Whenever I see Gaussian, I think of the Gauss gun from Half Life 2

I think in 2026 it's hard to make a video look this "bad" without it being a clear aesthetic choice, so not sure you could find this video in another setting.

The technology is impressive, but the end result… Weapons-grade brainrot.

I’m curious what other artists end up making with it.

Gaussian splatting is not NeRF (neural radiance field), but it is a type of radiance field, and supports novel view synthesis. The difference is in an explicit point cloud representation (Gaussian splatting), versus a process that needs to be inferred by a neural network.

> and then "NeRF"'d which means to extrapolate any missing data about their transposed 3D model

Not sure if it's you or the original article but that's a slightly misleading summary of NeRFs.

it does, it just doesn't have video

> Pretty sure most of this could be filmed with a camera drone and preprogrammed flight path

This is a “Dropbox is just ftp and rsync” level comment. There’s a shot in there where Rocky is sitting on top of the spinning blades of a helicopter and the camera smoothly transitions from flying around the room to solidly rotating along with the blades, so it’s fixed relative to rocky. Not only would programming a camera drone to follow this path be extremely difficult (and wouldn’t look as good), but just setting up the stunt would be cost prohibitive.

This is just one example of the hundreds you could come up with.

A drone path would not allow for such seamless transitions, never mind the planning required to nail all that choreography, effects, etc.

This approach is 100% flexible, and I'm sure at least part of the magic came from the process of play and experimentation in post.

If it was achievable, cheaper, and of equal quality then it would have been done that way. Surely it would’ve been done that way a long time ago too. Drone paths have been around a lot longer than this technology.

There’s no proof of your claim and this video is proof of the opposite.

Flying a camera drone with such proximity and acceleration would be a safety nightmare.

I think you’re missing the point

Volumetric capture like this allows you to decide on the camera angles in post-production

it gives you flexibility, options

It's fucking cool. That's why.

This tech is moving along at breakneck pace and now we're all talking about it. A drone video wouldn't have done that.

This might be the first time I'm stumbling on Dunning Kruger on HN, no offense.