Grain is independent frame-to-frame. It doesn't move with the objects in the scene (unless the video's already been encoded strangely). So long as the synthesized noise doesn't have an obvious temporal pattern, comparing stills should be fine.

Regarding aesthetics, I don't think AV1 synthesized grain takes into account the size of the grains in the source video, so chunky grain from an old film source, with its big silver halide crystals, will appear as fine grain in the synthesis, which looks wrong (this might be mitigated by a good film denoiser). It also doesn't model film's separate color components properly, but supposedly that doesn't matter because Netflix's video sources are often chroma subsampled to begin with: https://norkin.org/pdf/DCC_2018_AV1_film_grain.pdf

Disclaimer: I just read about this stuff casually so I could be wrong.

The noise does not contain a signal, does not dance over it, and is not detail. It is purely random fluctuations that are added to a signal.

If you have a few static frames and average them, you improve SNR by retaining the unchanged signal and having the purely random noise cancel itself out. Retaining noise itself is not useful.

I suspect the effect you might be seeing is either just an aesthetic preference for the original grain behavior, or that you are comparing low bandwidth content with heavy compression artifacts like smoothing/low pass filtering (not storing fine detail saves significant bandwidth) to high bandwidth versions that maintain full detail, entirely unrelated to the grain overlaid on top.

You could do an ai or compressed sensing upscale first, with multiple frames helping, have that be the base video sent, and then regranularize that.

> both it and the synthesized grain image look noticeably less sharp than the source

That's true, but at a given bitrate (until you get to very high bitrates), the compressed original will usually look worse and less sharp because so many bits are spent trying to encode the original grain. As a result, that original grain tends to get "smeared" over larger areas, making it look muddy. You lose sharpness in areas of the actual scene because it's trying (and often failing) to encode sharp grains.

Film Grain Synthesis makes sense for streaming where bandwidth is limited, but I'll agree that in the examples, the synthesized grain doesn't look very grain-like. And, depending on the amount and method of denoising, it can definitely blur details from the scene.

AV1 has tunable FGS levels, and to my eye they went very slightly higher than they should have (though there are tradeoffs; at some bitrates the blurring+renoising is so much better than the other visual artifacts you will otherwise get, that you do want it that high).

A few things to note:

- still-frames are also a mediocre way to evaluate video quality.

- a theoretically perfect[1] noise-removal filter will always look less detailed than the original source, since your brain/eye system will invent more detail for a noisy image than for a blurry image.

1: By which I mean a filter that preserves 100% of the non-grain detail present, not one that magically recovers detail lost due to noise.

Since the beginning of film editors have added tricks in post.

I would love for them to provide an option to view it with film simulation vs without.

One of my favorite movies of all time, The Holdovers, did film simulation extremely well. It's set in the '70s so it attempts to look like a movie of that era.

It looked great to me, but if you're an actual film nerd you're going to notice a lot of things aren't exactly accurate.

Maybe in the near future we'll see Netflix being able to process some post effects on the client. So if you're color blind, you get a mode for that. If you don't want fake grain you can turn it off.

A film that explores this is Antonioni’s Blowup: https://en.wikipedia.org/wiki/Blowup

Get the retro gaming nerds on this, they'll make a film grain shader that's indistinguishable from reality in no time flat.

Agree, as someone who has spent way too much time studying the way motion picture film looks up close, this isn’t very realistic looking. It’s really just a form of dithering.

> and not really grain like

that's an understatement. it just looks like RGB noise effect was added. film grain does not look like RGB noise. to me, film grain is only one part of what gave film the film look. the way the highlights bloom rather than clip. it also was more natural/organic/some descriptive other than the ultrasharp of modern digital acquisition. using some SoftFX or Black Mist type filters help, but it's just not the same as it is a digital vs analog type of acquisition. all of these attempts at making something look like it's not just keep falling down in the same ways. but hey, there's a cool tech blog about it this time. film grain filters have been around for a long time, yet people just don't care for them. even in Blu-ray time frame, there was attempts at removing the grain in the encode and applying it in playback. Netflix isn't coming up with anything new, and apparently nothing exciting either based on the results.

These days, when we see noise/grain in an end product it has likely been added in post-production. So, ideally, studios would provide distributors with a noiseless source plus grain synthesis parameters. Bonus: many viewers would welcome an option to turn it off.

My issue is that grain is good based on the creative decisions of the creators of the content. It is not something that a group of nerds compressing 1s and 0s should be making

The article points out the masking effect of grain, which hides the fake-looking compression artifacts, and also the familiarity/nostalgia aspect. But I will offer an additional explanation.

Look around you: nearly all surfaces have some kind of fine texture and are not visually uniform. When this is recorded as video, the fine texture is diminished due to things like camera optics, limited resolution, and compression smoothing. Film grain supplies some of the high frequency visual stimulus that was lost.

Our eyes and brains like that high frequency stimulation and aren't choosy about whether the exact noise pattern from the original scene is reproduced. That's why the x265 video encoder (which doesn't have grain synthesis since it produces H.265 video) has a psy-rd parameter that basically says, "try to keep the compressed video as 'energetic' as the original, even if the energy isn't in the exact same spot", and even a psy-rdoq parameter that says, "prefer higher 'energy' in general". These parameters can be adjusted to make a compressed video look better without needing to store more data.

People are always trying to rationalize and justify aesthetic preferences. The depth and nuance of your understanding of a thing will change how you perceive variations of that thing, whether it's guitar tonewoods, style of music, types of paint, flavor of beer, or the grain in film. If you know a lot about a subject, you can tell a lot about the history of a thing, and that's going to change how you feel about a thing.

A child watching a Buster Keaton skit and gasping and giggling and enjoying it is going to have a different subjective aesthetic experience of the media than a film critic who knows exactly what type of film and camera were used, and what the meaning of all the different abstractions imply about the scene, and the fabric of Keaton's costume, and so on, and so forth.

Subjective aesthetic preferences are in the realm of cognition - we need a formal theory of intelligence mapped to the human brain, and all of these subjective phenomena collapse into individualized data processing and initial conditions.

There's something about film grain contrasted against clean cel animation which might make it easier for people to suspend disbelief. They are conditioned to think that absence of grain is associated with unreal animation, particular types of media, and CGI. Home video and news and so forth had grain and low quality, so grain gets correlated with "real". In my view, there's nothing deeper than that - we're the product of our times. In 40 years, media will have changed, and it may be that film grain is associated with surrealism, or edited out completely, as it's fundamentally noise.

The way I see it is that grain makes the film look more detailed than it really is, it can also hide compression artefacts and blurriness.

I don't know the psychovisuals behind that. Maybe it adds some high frequencies that compression often washes out, or maybe acts like some kind of dithering.

As for your eyes, I am pretty sure that they have grain, that's how quantum physics work, you just don't perceive it because your brain filters it out. But again, I don't know how it interacts with film grain.

> my real eyes don’t have grain.

They definitely do at night when it's dark out. There's a kind of "sparkling" or "static" that comes in faint light.

Fortunately, our eyes have way better sensitivity than cameras. But the "realism" just comes from how it was captured using the technology of the day. It's no different from phonograph hiss or the way a CRT signal blurs. The idea is to be "real" to the technology that the filmmaker used, and the way they knew their movie would be seen.

It's the same way Van Gogh's brush strokes were real to his paintings. You wouldn't want his oil paintings sanded down to become flat. It's the reality of the original medium. And so even when we have a digital print of the film, we want to retain as much of the reality of the original as we can.

It's doubling back on itself. The film grain makes the footage look "cinematic" because it's how old movies looked.

Mine do, at least when it's very dark

It used to be a bigger deal (when digital cameras started being used) since people felt like digital video didn't look real/as good - movies shot on film were generally better looking (as crews were used shooting with it and digital video wasn't as sophisticated as today) and HAD grain.

It might be that there is a large part of the population that still has that association.

Cinephiles are also more likely to watch older (i.e. with grain) movies that ARE well shot and beautiful (which is why they are classics and watched by cinephiles) and not see bad film movies, only the cream of the crop, while being exposed to the whole gamut of quality when watching todays movies shot digitally. Would reinforce that grain = good while not being necessarily the case - and their opinion might be heard more than gen pop.

At any rate, it can be a neat tool to lower sharpness!

This reminds me of modern windows having fake panes. They’re just strips that are applied to give the impressions that there are multiple smaller panes because people are used to that and it feels “correct”.

I have to imagine past glassmakers would have been absolutely enthralled by the ability we now have to make uniform, large sheets of glass, but here we are emulating the compromises they had to make because we are used to how it looks.

My vision is grainy because of visual snow. Which is why I turn off film grain in games, it stacks on my vision and makes a visual mess.

Grain = realism because real captured grain isn't total random noise. It's authentic noisy data. It's part of captured scene. It adds subtle tiny but real detail to the scene. Unless I am corrected here and that real grain is also total random noise.

film grain adds realism in the same way that high frame rate films look wrong or vinyl sounds "warmer" or tube guitar amps sound "better" - It is what we are used to.

If your eyes did have grain, then it would still be applied to watching an "ungrained" film, as you're still using the same eyes.

Film grain can create stochastic resonance with the underlying ground truth. In practice, this can improve the perceived image quality over having none.

grain and 24fps and widescreen trigger certain contextual emotions around the movie-watching experience. remove them and your brain contextualizes the video very differently.

this is likely the result of ~100 years of film-based filmmaking and projection. hell, we still call it filmmaking.

Film grain and 24fps are both examples of people being far too attached to the technical limitations of film.

This sent me down a very interesting rabbit hole, thanks!

Fake film grain maybe, but saying all film grain needs to die is like saying oil painting brush strokes need to die.

> But there is no grain in reality, so it does the opposite

The author did not do a particularly good job of selling the merits of film grain. I am not sure what he meant by that but I will say that it does increase the perceived sharpness of the image as well as perceived detail, even if it is just an illusion. The cinematographer Steve Yedlin describes it as giving the viewer’s eyes something to “latch” onto.

Documentaries might care about accurately representing reality. For every other cinematic genre, "authenticity" is not an inherent goal. If film grain is part of the director's vision, then that's just as valid as a choice to have dramatic non-diagetic music playing in the background of a scene (which is highly inauthentic, but also highly effective at evoking emotion, which is the point of art).

> But there is no grain in reality, so it does the opposite

Well ackchually -- illumination is inherently random, so all time-bounded captures of a scene (including what your eyes do) are subject to shot noise: https://en.wikipedia.org/wiki/Shot_noise

> It just adds visual noise that obscures details of the authentic scene

Does it add any more than modern video compression techniques? What constitutes noise in cinema, is somewhat subjective.

You worry too much, it's all fake. What you think is the "footage" will be many layers from various sources composited together.

Fake lights, fake shadows, fake sky, ...

Only if you are aware of it, which 99.9% of people consuming video content are not. It’s simply an unimportant implementation detail (from a viewer’s perspective who doesn’t really care about bitrate-as-cost).

I think there is a bit more to it. For example when developing a game when CRTs were popular they were using CRTs to view their game and making decisions based on what they saw on the CRT. If you display the same game with perfect square pixels it looks different. If the developers were viewing square pixels when developing the game they would make different decisions.

Agree. Purely opining, but I assume that it's because of the emotional connection that artistic media has on people, despite the flaws.

People remember the emotions the artwork engendered, and thus the whole work is associated with the feelings, flaws and all. If the work is particularly widely known, the flaws can become a stand-in for the work itself.

I see this in video games - I'm fond of the NES-era "flaws" and limitations (palette limits, sprite limits, sound channel limits), but less connected to the Atari 2600 or SNES/PS1/NDS/etc flaws. Shovel Knight is charming; A Short Hike, while great, doesn't resonate on a style level.

Even if you ignore all of the emotional attachment aspects, I think you would expect this to happen simply because people who are good at their craft will learn how to work with the shortcomings of their tools. If your cameras unavoidably add grain to everything you shoot you'll find ways to make that look good, and some of the things you come up with might not work without grain.

For an example, watch Shogun. Director apparently thought that most of the screen being out of focus was a positive. Quite distracting.

This is just a fact of lossy compression: you want to throw away information that contributes less to the perception of the video so that you can describe it with fewer bits of information.

There are two possible advantages for this kind of grain synthesis. For Netflix, they could produce the same perceived quality at lower bitrates, which reduces costs per view and allows customers with marginally slow connections to get a higher quality version. For a consumer, the advantage would be getting more non-grain detail for a fixed bitrate.

You are right that if you subtract the dentists frame from the raw one, showing only the estimated noise, you would get some impression of the scene. I think there’s two reasons for this. Firstly, the places where the denoiser produced a blurry line that should be sharp may show up as faint lines. I don’t think this is ‘hidden information’ so much as it is information lost to lossy compression. In the same way, if you look at the difference between a raw image and one with compression, you may see some emphasized edges due to compression artefacts. Secondly, the less exposed regions of the film will have more noise so noisiness becomes a proxy for darkness, allowing some reproduction of the scene. I would expect this detail to be lost after adjusting for the piecewise linear function for grain intensity at different brightness levels.

Perhaps a third thing is the level of noise in the blacks and the ‘grain size’ or other statistical properties tell you about the kind of film being used, but I think those things are captured in the film grain simulation model.

Possibly there are some other artefacts like evidence of special effects, post processing, etc.

If you extract the grain (e.g. by subtracting a blurred version of the image), the result contains mostly noise, not meaningful scene information outside of some variation according to image brightness. Film grain is random, so the extracted "grain layer" doesn’t encode original image detail but film grain itself encodes relative lightness in its _density_.

A 4K/24p film encoded with Apple ProRes 4444 XQ (not even ProRes RAW) is 716 GB per hour, so you would need to swap a total of 30 Blu-ray discs once every 4 minutes in order to watch a 2 hour movie.

I wonder how much you'd get with such a technology... truly uncompressed 4K video you're talking about something on the order of a few terabytes for a 90-minute movie, so way way bigger than the biggest 4K Blu-ray discs. Lossless compression would get you under that number, but far enough to matter?

Your statement makes me think that one of the following is true:

1. You prefer Betamax or VHS to digital media (highly unlikely)

2. You own laserdiscs (limited to 480i)

3. You own 35mm prints of film.

Since all other formats film has been made available on are both digital media and compressed.

Uncompressed 4k video is ~5Gbps (3840 * 2160 * 3 * 24 * 8). A 2-hour movie clocks in at about 4.3TB. (3840 * 2160 * 3 * 24 * 60 * 60 * 2)

All that is 24fps.

That's without audio, which I assume you also want to be uncompressed.

agree

> How much grain is there in IMAX films?

I've read that a 15-perf 65mm IMAX negative shot with slower film stocks is "virtually grainless", even when viewed on a 70ft screen. Grain is apparently noticeable in IMAX films when large/fast stocks are used and pushed toward their limits, and (of course) when smaller-format film stocks have been blown up.

This is not fake grain - it's a reproduction of the original grain pattern, compressed separately from the underlying image content. The result is closer to the original picture than the denoised / compressed version.

There's no need for low-quality comments cutting and pasting from your favourite LLM. You can literally just type it into your browser's search bar and learn what it is, or spend a minute reading the wiki page (https://en.wikipedia.org/wiki/AV1).