Funnily enough that's not the only mistake he made in that article. His final image is noticeably different from the camera's output image because he rescaled the values in the first step. That's why the dark areas look so crushed, eg around the firewood carrier on the lower left or around the cat, and similarly with highlights, e.g. the specular highlights on the ornaments.

After that, the next most important problem is the fact he operates in the wrong color space, where he's boosting raw RGB channels rather than luminance. That means that some objects appear much too saturated.

So his photo isn't "unprocessed", it's just incorrectly processed.

If someone's curious about those particular constants, they're the PAL Y' matrix coefficients: https://en.wikipedia.org/wiki/Y%E2%80%B2UV#SDTV_with_BT.470

> The quantization formula ppmtopgm uses is g = .299 r + .587 g + .114 b.

Seriously. We can trust linux man pages to use the same 1-letter variable name for 2 different things in a tiny formula, can't we?

> we are most sensitive to red light

> green does indeed overwhelmingly contribute to perceptual luminance

so... if luminance contribution is different from "sensitivity" to you - what do you imply by sensitivity?

Is it related to the fact that monkeys/humans evolved around dense green forests ?

I wouldn't worry about it too much, looking at ads is always a shitty experience. Correctly grayscaled or not.

I don't think Kindle ads were available in my region in 2015 because I don't remember seeing these back then, but you're a lucky one to fix this classic mistake :-)

I remember trying out some of the home-made methods while I was implementing a creative work section for a school assignment. It’s surprising how "flat" the basic average looks until you actually respect the coefficients (usually some flavor of 0.21R + 0.72G + 0.07B). I bet it's even more apparent in a 4-bit display.

True, but there may be different intentions behind the processing.

Sometimes the processing has only the goal to compensate the defects of the image sensor and of the optical elements, in order to obtain the most accurate information about the light originally coming from the scene.

Other times the goal of the processing is just to obtain an image that appears best to the photographer, for some reason.

For casual photographers, the latter goal is typical, but in scientific or technical applications the former goal is frequently encountered.

Ideally, a "raw" image format is one where the differences between it and the original image are well characterized and there are no additional unknown image changes done for an "artistic" effect, in order to allow further processing when having either one of the previously enumerated goals.

Exactly - film photographers heavily process(ed) their images from the film processing through to the print. Ansel Adams wrote a few books on the topic and they’re great reads.

And different films and photo papers can have totally different looks, defined by the chemistry of the manufacturer and however _they_ want things to look.

Thanks! First hit online: https://www.lab404.com/3741/readings/sontag.pdf

Out of curiosity: what led you to write "Susan Sunday" instead of "Susan Sontag"? (for other readers: "Sonntag" is German for "Sunday")

Great series of articles!

Protanopia and protanomaly shift luminance perception away from the longest wavelengths of visible light, which causes highly-saturated red colours to appear dark or black. Deuteranopia and deuteranomaly don't have this effect. [1]

Blue cones make little or no contribution to luminance. Red cones are sensitive across the full spectrum of visual light, but green cones have no sensitivity to the longest wavelengths [2]. Since protans don't have the "hardware" to sense long wavelengths, it's inevitable that they'd have unusual luminance perception.

I'm not sure why deutans have such a normal luminous efficiency curve (and I can't find anything in a quick literature search), but it must involve the blue cones, because there's no way to produce that curve from the red-cone response alone.

[1]: https://en.wikipedia.org/wiki/Luminous_efficiency_function#C...

[2]: https://commons.wikimedia.org/wiki/File:Cone-fundamentals-wi...

The cones are the colour sensitive portion of the retina, but only make up a small percent of all the light detecting cells. The rods (more or less the brightness detecting cells) would still function in a deuteranopic person, so their luminance perception would basically be unaffected.

Also there’s something to be said about the fact that the eye is a squishy analog device, and so even if the medium wavelengths cones are deficient, long wavelength cones (red-ish) have overlap in their light sensitivities along with medium cones so…

It’s not that their M-cones (middle, i.e. green) don’t work at all, their M-cones responsivity curve is just shifted to be less distinguishable from their L-cones curve, so they effectively have double (or more) the “red sensors”.

Yeah, was thinking the same. If we're more sensitive, why do we need double sensors? Just have 1:1:1, and we would anyways see more of the green? Won't it be too much if we do 1:2:1, when we're already more perceptible to green?

Since the human eye is most sensitive to green, it will find errors in the green channel much easier than the others. This is why you need _more_ green data.

Note that there are two measurement systems involved: first the camera, and then the human eyes. Your reasoning could be correct if there were only one: "the sensor is most sensitive to green light, so less sensor area is needed".

But it is not the case, we are first measuring with cameras, and then presenting the image to human eyes. Being more sensitive to a colour means that the same measurement error will lead to more observable artifacts. So to maximize visual authenticity, the best we can do is to make our cameras as sensitive to green light (relatively) as human eyes.

This ratio allows for a relatively simple 2x2 repeating pattern. That makes interpolating the values immensely simpler.

Also you don't want the red and blue to be too far apart, reconstructing the colour signal is difficult enough as it is. Moire effects are only going to get worse if you use an even sparser resolution.

As a mostly amateur photographer, it doesn't bother me if people ask that question. While I understand the point that the camera itself may be making some 'editing' type decision on the data first, a) in theory each camera maker has attempted to calibrate the output to some standard, b) public would expect two photos taken at same time with same model camera should look identical. That differs greatly from what often can happen in "post production" editing - you'll never find two that are identical.

I don't think it's the same, for me personally I don't like heavily processed images. But not in the sense that they need processing to look decent or to convey the perception of what it was like in real life, more in the sense that the edits change the reality in a significant way so it affects the mood and the experience. For example, you take a photo on a drab cloudy day, but then edit the white balance to make it seem like golden hour, or brighten a part to make it seems like a ray of light was hitting that spot. Adjusting the exposure, touching up slightly, that's all fine, depending on what you are trying to achieve of course. But what I see on instagram or shorts these days is people comparing their raws and edited photos, and without the edits the composition and subject would be just mediocre and uninteresting.

I noticed this a lot when taking pictures in the mountains.

I used to have a high resolution phone camera from a cheaper phone and then later switched to an iPhone. The latter produced much nicer pictures, my old phone just produces very flat-looking pictures.

People say that the iPhone camera automatically edits the images to look better. And in a way I notice that too. But that’s the wrong way of looking at it; the more-edited picture from the iPhone actually corrresponds more to my perception when I’m actually looking at the scene. The white of the snow and glaciers and the deep blue sky really does look amazing in real life, and when my old phone captured it into a flat and disappointing looking photo with less postprocessing than an iPhone, it genuinely failed to capture what I can see with my eyes. And the more vibrant post processed colours of an iPhone really do look more like what I think I’m looking at.

JPEG with OOC processing is different from JPEG OOPC (out-of-phone-camera) processing. Thank Samsung for forcing the need to differentiate.

There's a difference between an unbiased (roughly speaking) pipeline and what (for example) JBIG2 did. The latter counts as "editing" and "fake" as far as I'm concerned. It may not be a crime but at least personally I think it's inherently dishonest to attempt to play such things off as "original".

And then there's all the nonsense BigTech enables out of the box today with automated AI touch ups. That definitely qualifies as fakery although the end result may be visually pleasing and some people might find it desirable.

it's not a crime but applying post processing in an overly generous way that goes a lot further than replicating what a human sees does take away from what makes pictures interesting imho vs other mediums, that it's a genuine representation of something that actually happened.

if you take that away, a picture is not very interesting, it's hyperrealistic so not super creative a lot of the time (compared to eg paintings), & it doesn't even require the mastery of other mediums to get hyperrealistism

Green is in the center of the visible spectrum of light (notice the G in the middle of ROYGBIV), so evolution should theoretically optimize for green light absorption. An interesting article on why plants typically reflect that wavelength and absorb the others: https://en.wikipedia.org/wiki/Purple_Earth_hypothesis

Several reasons, -Silicon efficiency (QE) peaks in the green -Green spectral response curve is close to the luminance curve humans see, like you said. -Twice the pixels to increase the effective resolution in the green/luminance channel, color channels in YUV contribute almost no details.

Why is YUV or other luminance-chrominance color spaces important for a RGB input? Because many processing steps and encoders, work in YUV colorspaces. This wasn't really covered in the article.

You think that's bad? Imagine finding out that all video still encodes colour at half resolution simply because that is how analog tv worked.

Not sure why it would invoke such strong sentiments but if you don’t like the bayer filter, know that some true monochrome cameras don’t use it and make every sensor pixel available to the final image.

For instance, the Leica M series have specific monochrome versions with huge resolutions and better monochrome rendering.

You can also modify some cameras and remove the filter, but the results usually need processing. A side effect is that the now exposed sensor is more sensitive to both ends of the spectrum.

If the Bayer pattern makes you angry, I imagine it would really piss you off to realize that the whole concept encoding an experienced color by a finite number of component colors is fundamentally species-specific and tied to the details of our specific color sensors.

To truly record an appearance without reference to the sensory system of our species, you would need to encode the full electromagnetic spectrum from each point. Even then, you would still need to decide on a cutoff for the spectrum.

...and hope that nobody ever told you about coherence phenomena.

Upon inspection, the author's personal website used em dashes in 2023. I hope this helped with your witch hunt.

I'm imagining a sort of Logan's Run-like scifi setup where only people with a documented em dash before November 30, 2022, i.e. D(ash)-day, are left with permission to write.

I have been overusing em dashes and bulleted lists since the actual 80s, I'm sad to say. I spent much of the 90s manually typing "smart" quotes.

I have actually been deliberately modifying my long-time writing style and use of punctuation to look less like an LLM. I'm not sure how I feel about this.

found the guy who didn't know about em dashes before this year

also your question implies a bad assumption even if you disclaim it. if you don't want to imply a bad assumption the way to do that is to not say the words, not disclaim them

It's like food: Virtually all food is "processed food" because all food requires some kind of process before you can eat it. Perhaps that process is "picking the fruit from the tree", or "peeling". But it's all processed in one way or another.

It's not really - it's going in the inverse direction regarding how much more processed and artificially altered it gets.

Except it seems to be going in the opposite direction, every phone I've upgraded (various Androids and iPhones) seemed to have more smoothing than the one I'd had before. My iPhone 16 night photos look like borderline generative AI and there's no way to turn that off!

I was honestly happier with the technically inferior iPhone 5 camera, the photos at least didn't look fake.

"Better"...

Huh wonder which camera apps enable use of this API?

This is VERY interesting and I am glad you posted this, as it is my first time coming across this.

> What most people think when they say "fake", though, is a photo that has had filters applied, which makes zero sense. As the post shows, all photos have filters applied.

Filters themselves don't make it fake, just like words themselves don't make something a lie. How the filters and words are used, whether they bring us closer or further from some truth, is what makes the difference.

Photos implicitly convey, usually, 'this is what you would see if you were there'. Obviously filters can help with that, as in the OP, or hurt.

There's an obvious difference between debayering and white balance vs using Photoshop's generative fill

> "Fake" images are images with intent to deceive

The ones that make the annual rounds up here in New England are those foliage photos with saturation jacked. “Look at how amazing it was!” They’re easy to spot since doing that usually wildly blows out the blues in the photo unless you know enough to selectively pull those back.

Everything is an interpretation of the data that the camera has to do

What about this? https://news.ycombinator.com/item?id=35107601

Well that's why back in the day (and even still) 'Photographer listing their whole kit for every shot' is a thing thing you sometimes see.

i.e. Camera+Lens+ISO+SS+FStop+FL+TC (If present)+Filter (If present). Add focus distance if being super duper proper.

And some of that is to help at least provide the right requirements to try to recreate.

A boss once asked me "is there a way to tell if an image has been Photoshopped?" and I did eventually get him to "yes, if you can see the image it has been digitally processed and altered by that processing". (The brand-name-as-generic conversation was saved for another day.)

> A better discriminator might be global edits vs local edits,

Even that isn't all that clear-cut. Is noise removal a local edit? It only touches some pixels, but obviously, that's a silly take.

Is automated dust removal still global? The same idea, just a bit more selective. If we let it slide, what about automated skin blemish removal? Depth map + relighting, de-hazing, or fake bokeh? I think that modern image processing techniques really blur the distinction here because many edits that would previously need to be done selectively by hand are now a "global" filter that's a single keypress away.

Intent is the defining factor, as you note, but intent is... often hazy. If you dial down the exposure to make the photo more dramatic / more sinister, you're manipulating emotions too. Yet, that kind of editing is perfectly OK in photojournalism. Adding or removing elements for dramatic effect? Not so much.

Eh, I'm a photographer and I don't fully agree. Of course almost all photos these days are edited in some form. Intent is important, yes. But there are still some kinds of edits that immediately classify a photo as "fake" for me.

For example if you add snow to a shot with masking or generative AI. It's fake because the real life experience was not actually snowing. You can't just hallucinate a major part of the image - that counts as fake to me. A major departure from the reality of the scene. Many other types of edits don't have this property because they are mostly based on the reality of what occurred.

I think for me this comes from an intrinsic valuing of the act/craft of photography, in the physical sense. Once an image is too digitally manipulated then it's less photography and more digital art.

Today, I trust the other meaning of "fake images" is that an image was generated by AI.

But when you shift the goal posts that far, a real image has never been produced. But people very clearly want to describe when an image has been modified to represent something that didn’t happen.

I understand what you and the article are saying, but what GP is getting at, and what I agree with, is that there is a difference between a photo that attempts to reproduce what the "average" human sees, and digital processing that augments the image in ways that no human could possibly visualize. Sometimes we create "fake" images to improve clarity, detail, etc., but that's still less "fake" than smoothing skin to remove blemishes, or removing background objects. One is clearly a closer approximation of how we perceive reality than the other.

So there are levels of image processing, and it would be wrong to dump them all in the same category.

Don't you think you're being a bit too pedantic? Nothing really "sees". Eyes also do gather light and the brain analyzes the signals. We've invented a word for it but the word is a high level abstraction that could easily be applied to a camera sensor as well.

Hey thanks a bunch. These were not the ones though. The blog was more mathy and had a signal processing analysis and fixed point analysis of the repeated blue and sharpen phenomena.

I had upvoted it on HN either as a post or a comment that had the link. Wish there was an easy way to search through ones own upvoted comments and posts.

Thanks again though for trying to help.

The shape of the curve doesn't matter at all. What matters is having a mismatch between the capture curve and the display curve.

If you kept it linear all the way to the output pixels, it would look fine. You only have to go nonlinear because the screen expects nonlinear data. The screen expects this because it saves a few bits, which is nice but far from necessary.

To put it another way, it appears so dark because it isn't being "displayed directly". It's going directly out to the monitor, and the chip inside the monitor is distorting it.

>Human light intensity perception is not linear... You have to nonlinearize at some point of the pipeline

Why exactly? My understanding is that gamma correction is effectively a optimization scheme during encoding to allocate bits in a perceptually uniform way across the dynamic range. But if you just have enough bits to work with and are not concerned with file sizes (and assuming all hardware could support these higher bit depths), then this shouldn't matter? IIRC unlike crts, LCDs don't have a power curve response in terms of the hardware anyway, and emulate the overall 2.2 trc via LUT. So you could certainly get monitors to accept linear input (assuming you manage to crank up the bit depth enough to the point where you're not losing perceptual fidelity), and just do everything in linear light.

In fact if you just encoded the linear values as floats that would probably give you best of both worlds, since floating point is basically log-encoding where density of floats is lower at the higher end of the range.

https://www.scantips.com/lights/gamma2.html (I don't agree with a lot of the claims there, but it has a nice calculator)

Linear encoding doesn't change the max brightness of the monitor.

More importantly, the camera isn't recording blinding brightness in the first place! It'll say those pixels are pure white, which is probably a few hundred or thousand nits depending on shutter settings.

That's a matter of tone mapping which is separate from gamma encoding? Even today, linearized pixel value 255 will be displayed at your defined SDR brightness no matter what. Changing your encoding gamma won't help that because for correct output the transform necessarily needs to be be undone during display.

Which is why I'm looking at replacing my car's rear-view mirror with a camera and a monitor. Because I can hard-cap the monitor brightness and curve the brightness below that, eliminating the problem of billion-lumens headlights behind me.

Not a complaint, but both the final images have poor contrast, lighting, saturation and colour balance, making them a disappointing target for an explanation of how these elements are produced from raw sensor data.

But anyway, I enjoyed the article.

Not quite. At the very least, the RAW file is a static file. Whereas your camera chip can make interactive decisions.

In any case, RAW files aren't even all that raw. First, they are digitised. They often apply de-noising, digital conditioning (to take care of hot and dead pixels), lens correction. Some cameras even apply some lossy compression.

My mirrorless camera shoots in RAW. When someone asks me if a certain photo was “edited”, I honestly don’t know what to answer. The files went through a RAW development suite that applied a bewildering amount of maths to transform them into a sRGB image. Some of the maths had sliders attached to it and I have moved some of the sliders, but their default positions were just what the software thought was appropriate. The camera isn’t even set to produce a JPEG + RAW combo, so there is literally no reference.

What about taking 3 photos while quickly changing the filter (e.g. filters are something like quantum dots that can be turned on/off)?

B&W sensors are generally more sensitive than their color versions, as all filters (going back to signal processing..) attenuate the signal.

So, for a 3x3 image, the input data would be 9 values like:

   R G B
   B R G
   G B R

?

I said “flat” because I didn’t feel like going into “log” and color profiles and such but I’ll admit I’m leaning hard into over-simplification, because log, raw, etc. gets messy when discussing profiles vs codecs/compression/etc. In video we still call some codecs “raw,” but it’s not the same necessarily as how it’s used in photography. Like the Red raw codec has various compression ratios (5:1 tends to be the sweet spot IME) and it really messes with the whole idea of what raw even is. It’s all quasi-technical and somewhat inconsistent.

My point wasn't "I can't find this information", my point was "this is poorly written".

Sure, and my camera can do bird eye detection and whatnot too, but that's a very lightweight model running in-body. Probably just a fine-tuned variant of something like YOLO.

I've seen only a couple of papers from Google talking about stacking multiple frames from a DSLR, but that was only research for improving mobile phone cameras.

Ironically, some mobile phones now have more megapixels than my flagship full-frame camera, yet they manage to stack and digitally process multiple frames using battery power!

This whole thing reminds me of the Silicon Graphics era, where the sales person would tell you with a straight face that it's worth spending $60K on a workstation and GPU combo that can't even texture map when I just got a Radeon for $250 that runs circles around it.

One industry's "impossible" is a long-since overcome minor hurdle for another.

I trawled through much of the research but as you’ve mentioned it seems to be known only in astrophotography and mobile devices or other similarly constrained hardware.