I think that might be why a lot of camera makers don't care to use DNG - it's easier to make their own format and easy enough for others to reverse engineer it.

One thing that open source libraries do tend to miss is that very important extra metadata - for example, Phase One IIQ files have an embedded sensor profile or full on black frame that is not yet encoded into the raw data like it typically is for a NEF or DNG from many cameras. It does seem rawspeed handles this from a quick scan of the code.

It can get more tricky - Sinar digital backs have an extra dark frame file (and flat frame!) that is not part of the RAW files, and that is not handled by any open source library to my knowledge - though I did write a basic converter myself to handle it: https://github.com/mgolub2/iatodng_rs

I'm not sure how DNG would be able to handle having both a dark and flat frame without resorting to applying them to the raw data and saving only the processed (still unbayered) data.

It took a long time for Canon CR3 raw format to be supported by darktable because, although the format itself had been reverse engineered, there was a fear from the developers that it was covered by a patent and that they risked a lawsuit by integrating it in DT. IIRC, they had attempted to contact Cabon legal to obtain some sort of waiver, without success.

I'm fact I'm not sure how that saga ended and CR3 support was finally added a few years after the release of the Canon mirrorless cameras that output CR3.

> Cameras absolutely could emit DNG instead, but that would require more development friction: coordination (with Adobe), [..]

Technically speaking, implementing DNG would be another development activity on top of a RAW export, because RAW also has a purpose in development and tuning of the camera and its firmware.

It is supposed to be raw data from the sensor with some additional metrics streamed in, just sufficiently standardized to be used in the camera-vendors' toolchain for development.

It just "happens" to be also available to select for the end-user after product-launch. Supporting DNG would mean adding an extra feature and then hiding the RAW-option again.

I can imagine it's hard to make this a priority in a project plan, since most of the objectives are already achieved by saving in RAW

Fujifilm lossy compressed raw still isn't supported after many years [1].

[1] https://github.com/darktable-org/rawspeed/issues/366

And in my experience there has been lots of bugs with Fujifilm raws in darktable:

[2] https://github.com/darktable-org/rawspeed/issues/354

[3] https://github.com/darktable-org/darktable/issues/18073

However, Fujifilm lossless compressed raw actually does a decent job keeping the file sizes down (about 50% to 60% the file size of uncompressed) while maintaining decent write speed during burst shooting.

When my father bought his first digital camera, he insisted on raw format access.

Inexplicably I didn't understand at the time why he (Bryce Bayer) wanted this. He was modest about his work.

I do now!

I always thought camera RAW formats were optimize continuous shooting rates. About being able to linearly write an image as fast as possible.

I don't know the details of DNG but even the slightest complication could be a no-go for some manufacturers.

The contents are simple. How to interpret the contents, is not simple. That is why you see internet advice advocating for keeping old raw files around, because Lightroom and Photoshop sometimes gets updates which can cram out better results from old raw files.

(Edit: I mean, if you want to get a basic debayered RGB image from a raw, that's not too hard. But if you want to cram out the most, there are a lot of devils in a lot of details. Things like estimating how many green pixels are not actually green, but light-spill from what should have been red pixels is just the beginning.)

> Cameras absolutely could emit DNG instead, but that would require more development friction: coordination (with Adobe), potentially a language barrier, and potentially making it harder to do experimental features.

I think this is being too generous.

DNG is just an offshoot of TIFF. Having written a basic DNG parser having never read up on TIFFs before, it really isn’t that hard.

As far as experimental features, there’s room in the spec for injecting your own stuff, similar to MakerNote in EXIF if I recall.

If you are planning to do experimental stuff, I’d say what Apple pulled off with ProRAW is the most innovative thing that a camera manufacturer has done in forever. They worked with Adobe to get it into the spec. All of these camera manufacturers have similar working relationships with Adobe, so there’s really no excuse. And if you can’t wait that long, again, MakerNote it.

In my opinion, custom RAW formats are a case study in “Not Invented Here” syndrome.

These formats aren't complex because they really are supposed to be raw (-:

But yeah, it would be preferable to have them use the digital negative (DNG) format, but why bother when the community makes the work for them? Reminds me of how Bethesda does things.

> Cameras absolutely could emit DNG instead, but that would require more development friction: coordination (with Adobe), potentially a language barrier, and potentially making it harder to do experimental features.

I am a weirdo and always liked and used Pentax (now Ricoh) they do support the DNG format.

Why would you need coordination with Adobe? Their software already reads DNG files just fine.

> Cameras absolutely could emit DNG instead, but that would require more development friction: coordination (with Adobe), potentially a language barrier, and potentially making it harder to do experimental features

I've worked with medical imaging systems from the largest imaging companies in the world -- GE, Siemens, etc. -- all of which use a standardized image format/protocol/etc. called DICOM. DICOM has standardized fields for the vast majority of information you would need to record for medical imaging - patient ID, study ID, image # if it's an image sequence, etc. - as well as metadata about where it came from, like the vendor ID of the machine that did the scan (the CT scanner, MRI, X-ray, etc). There are also arbitrary fields for vendor-specific information that doesn't have a defined field in the specification.

All of these fields have clear purposes and definitions and all are available to every DICOM reader/writer, and yet the company I worked for had a huge table of re-mappings because some scanners, for some reason, would put the patient ID in the vendor field, or the vendor ID in the scanner name field, and so on. There's no reason for this, there's no complication that might cause this; it's all standard fields that everything supports.

These are manufacturers who, while using the standard that everyone else uses, deliberately screw things up in ways that their own hardware and software can silently compensate for but which other vendors then have to work around in order to inter-operate.

In other words cameras absolutely could emit DNG instead, but aside from the arguments that you've made, I have every confidence that manufacturers would fuck it up on purpose just to make it harder for other vendors' software to inter-operate, which would mean that instead of software having to explicitly support e.g. Canon's RAW format, and being able to say "we don't yet support this new format", software would "support" DNG but it would be completely broken for some random cameras because the software developer hasn't had the chance to implement idiotic workarounds for these specific broken images yet.

Well, it is obvious that between a RAW file and the final image there are a lot of complex processing steps. But that is independent of the file format used. DNG isn't so much different, just documented. And while the manufacturers converter might give the best results, the photographers rather use the image processing programs from Adobe or their competition which use their own RAW converters anyway.

Anecdotally, using Darktable, I could never get as good of a demosaicing result as using the straight-out-of-camera JPEGs from my Fujifilm GFX 100S. In challenging scenarios such as fine diagonal lines, Darktable's algorithms such as LMMSE would add a lot of false colour to the image.

However, modern deep learning-based joint demosaicing and denoising algorithms handily outperform Darktable's classical algorithms.

But this is talking about proprietary RAW image formats, which should be the data from the sensor with minimal processing. The entire point of RAW images is that you are skipping the majority of the ISP and applying those blocks in the processing software. Even the de-mosiacing step is done in the processor. There is really no reason why this is proprietary. This doesn't stop the camera company from applying their proprietary processing on the JPEG output that utilizes the full ISP.

Can you share what company have you worked for?

Raw decoding is an algorithm, not a container format. The issue is every is coming up with their own proprietary containers for identical data that just represents sensor readings.

What does that have to do with storing the raw sensor values?

> Camera sensors from different companies (and different generations) don't have the same color (or if you prefer, spectral) responses with both their Bayer filter layer and the underlying physical sensor

This is all accommodated for in the DNG spec. The camera manufacturers specify the necessary matrix transforms to get into the XYZ colorspace, along with a linearization table.

If they really think the spectral sensitivity is some valuable IP, they are delusional. It should take one Macbeth chart, a spreadsheet, and one afternoon to reverse engineer this stuff.

Given that third party libraries have figured this stuff out, seems they have failed while only making things more difficult for users.

What does RAW really mean then? Couldn't they simply redefine what RAW means to create a standard that can include proprietary technology? Like why not define it as including a spectral response?

> But in theory the RAW converter has been profiled for each specific camera using ICC color targets and stuff like that.

In practice too, if consistent results are desired. The format being identical doesn't mean the values the sensor captures under the same conditions will be identical, so a color-calibrated workflow could produce wrong results.

It would be nice to have a setting for "treat camera Y like camera X (here there be dragons)" though. I've had to do something similar with the Lensfun database to get lens corrections working on Mk. II of a lens where Mk. I was supported, but a GUI would be nice. A prompt to guess the substitution automatically would be even nicer.

Most hardware companies are just terrible at software in general. Camera makers are pretty average in that regard.

Usability of the camera hardware and software ecosystem is another matter. I think the common wisdom is that most paying users don't want beginner-friendly, they want powerful and familiar. So everything emulates the paradigms of what came before. DSLRs try to provide an interface that would be familiar to someone used to a 50 year old SLR camera, and Lightroom tries to emulate a physical darkroom. Being somewhat hostile to the uninitiated might even be seen as a feature.

Over the past 15-20 years I've used both Sonys, Canons and Nikons, and I absolutely feel that Nikon puts a lot more effort, with much better results, into the usability of their pro/prosumer cameras - and, really, even their $500-$1000 consumer range - both in terms of the on-display UI and the ergonomics and handling of the actual camera.

What always stood out most for me compared to Canon was Nikon's larger viewfinders, letting you commit to actual photography rather than being stuck with a feeling of peeping through a keyhole, and placement of buttons on the camera body allowing for maintained control of the most necessary functions (shutter speed, aperture and even ISO) without having to change your grip or move the camera away from your face.

Sony is famous for having the worst interfaces of all the big camera manufacturers.

Lightroom most likely has “obsolete paradigms” for the same reason Photoshop does: because professionals want to use what they know rather than what is fashionable. Reprogramming their muscle memory is not something people want to be doing. Anyway, I find Lightroom’s UI very nice to work with.

Yes, same for my Sony A6000 and A6400. I just wanted to take some selfies and it's exhausting to use the remote app.

Fujifilm is no better haha, they're on their third (I think) mobile app this time for X-Series camera control, and it's still terrible.'

Amusingly, their Instax control app is actually pretty good!

Things like camera intrinsics and extrinsics are not fixed. 1000 bytes seems small to me given the amount of processing in modern cameras to create a raw image. I could easily imagine storing more information like focus point, other potential focus points with weights as part of the image for easier user on device editing.

For testing, there's the Adobe DNG SDK: https://helpx.adobe.com/camera-raw/digital-negative.html

You'll find the whole spec there, too. I think the source is also available somewhere.

The camera app on my GNU/Linux phone stores DNGs with no troubles using FLOSS only.

Seems like DNG does behave like the RAW lite format you've just described: Everything common would be stored within the base DNG file, while everything "advanced" / more specific to a camera would be stored in additional metadata properties, which do not need to be parsed to still be able to process the base image. You can add support for these metadata on a case-by-case basis without breaking the original format, so you're not stuck re-implementing your whole raw parsing when a new camera is released as the base subset of DNG would still work.

Hey. I’m the guy quoted.

RAW is ultimately about sensor readings. As a developer, you just want to get things from there into a linear, known color space (XYZ in the DNG spec). So from that perspective, interoperability isn’t the issue.

How you process that data is another matter. Handling a traditional bayer pattern vs a quad-bayer vs Fujifilm’s x-trans pattern obviously requires different algorithms, but that’s all moot given DNG is just a container.

Most image sensors are quite similar (ignoring weirdos like X-Trans and Foveon) so they could use the same format and decoding algorithm. It's a 16-bit integer (padded out from 12 or 14 bits) for each pixel with a Bayer color filter. Maybe throw in some parameters like a suggested gamma curve.

Typically they are not to my knowledge! Though I am also not an expert. Most camera makers apply a fixed sensor profile, and possibly a dark frame to remove noise before writing out the values to whatever file. Some of them may apply lens optimizations to correct distortion or vignetting as well.

It's all float value arrays with metadata in the end. Most camera sensors are pretty similar and follow common patterns.

DNGs have added benefits, like including compression (optional) (either lossy or lossless) and error correction bytes to prevent corruption (optional). Even if there's some concerns like unique features or performance, I'd still rather use DNGs without these features and with reduced performance.

I always archive my RAWs as lossy compressed DNGs with error correction and without thumbnails to save space and some added "safety".

The ironic part is, that basically all the closed-source photo-editing software (and of course all open-source) are just using the open source LibRaw. Any special features as color profiles comes on top of that. So yes, the camera manufacturers could just donate to LibRaw or just use DNG instead.

I downloaded a sample RAF file from the internet. It was 16 megapixels, and 34MB in size. I used Adobe DNG Converter, and it created a file that was 25MB in size. It was actually smaller.

Claiming that DNG takes up 4x space doesn't align with any of my own experiences, and it didn't happen on the RAF file that I just tested.

To test this, I downloaded a random RAF image from this gallery https://mirrorlesscomparison.com/galleries/fuji-xt2-sample-i...

    $ /Applications/Adobe\ DNG\ Converter.app/Contents/MacOS/Adobe\ DNG\ Converter DSCF6001.RAF
    SPL-LOG-1002: starting logger thread
    *** GPU Warning: GPU3 disabled via cr_config at init time. ***
    SPL-LOG-1003: terminating logger thread
    SPL ~DefaultMemoryManagerImpl bytesAllocated = 0
    $ ls -la DSC*
    -rw-r--r-- 1 danielh staff 50377216 2025-04-07T10:59:30 DSCF6001.RAF
    -rw-r--r-- 1 danielh staff 30747896 2025-04-07T11:00:13 DSCF6001.dng

No it doesn't. A DNG is just a container that can hold anything, including compressed data, just like the RAF.

Coincidentally, most proprietary RAW formats are just bastardized TIFFs, and DNG is also a TIFF derivative...

There is zero technical reason not to use DNG. Leica and Pentax use it just fine.

Proprietary formats require 3rd party developers to adapt their tools: While most mainstream software will be updated to support most/all cameras, this makes it harder for smaller projects to do. If they used an open standard, the advanced features could still require additional work to be compatible (ex: If they store custom metadata), but you could normalize everything that's shared, ensuring the core capabilities will never break for a new camera with its updated proprietary RAW like it currently does.

> Is raw a defined format?

It's not /dev/urandom written to disk, no. Yes, a raw format has a structure. There's not one "RAW" format though (and TFA notes this): e.g., my Canon's RAW format specifically referred to is called "CR3". And its predecessor was "CR2", so even within a manufacturer there are multiple such formats. All undocumented.

But a Pentax won't write out CR3s, it'll write out some other, yet bespoke format.

I've vaguely reverse engineered some of CR3: it is a container that contains multiple copies of the photo taken; IIRC it contains a thumbnail JPEG, the JPEG, and the raw data itself.

I doubt it's the most performant to write to storage: the format is vaguely TLV (its fairly similar to RIFF, if you understand the RIFF format), so it can't really be streamed to storage due to needing to know the lengths of the containing chunks (all the way out to the outermost chunk).

> This is confusing.

We’ll, yes. You’re thinking of the classic RAW format that was just a plain array of RGB pixels without a header.

When talking about digital cameras RAW refers to a collection of vendor specific file formats for capturing raw sensor data, together with a bunch of metadata.

> what order the 3 colors are in (RGB or BGR)

Camera raw files typically come in a raw bayer mosaic so each pixel has only one colour.

There’s DICOM but last I worked with it it was a disaster. , because everyone implemted it differently. Seemed like a solution looking for a problem. Also didn’t help the standard is like 200 pages and it was a design by committee, none of whom appeared actually interested in coming up with a reference implementation.

I never understood why everyone didn’t just use TIFF

Sigma currently doesn't produce any Foveon cameras - the fp, fp L and BF all use DNG.

> And in a lot of conditions, my Android phone takes better photos, by virtue of more advanced technology.

> I have tens of thousands of dollars of camera equipment -- mostly more than a decade old -- and there just haven't been advancements warranting an upgrade. A modern camera will be maybe 30% better than a 2012-era one in terms of image quality, and otherwise, will have slightly more megapixels, somewhat better autofocus, and obviously be much smaller by the loss of a mirror. Video improved too.

I thought the same thing, and then I went and rented a Nikon Z8 to try out over a weekend and I was blown away by the "somewhat better autofocus". As someone who used to travel with a Pelican case full of camera gear, to just carrying an iPhone, I'm back to packing camera gear because I'm able to do things like capture tack-sharp birds in flight like I'm taking snapshots from the hip thanks to the massive increase in compute power and autofocus algorithms. "Subject Eye Detection AF" is a game-changer, and while phones do it, they don't have enough optical performance in their tiny sensors/lenses to do it at the necessary precision and speed to resolve things on fast-moving subjects.

In terms of IQ, weight, and all that, it's definitely not a huge difference. I would say it's better, but not so much that I particularly cared coming from a 12-year old DSLR. But the new AF absolutely shocked me with how good it is. It completely changed my outlook.

I say this, not to take away from your overall point, however, which is that a phone is good enough for almost everyone about 90% of the time. It's good enough that even though I upgraded my gear, I only bought one body when I traded in two, because my phone can handle short-focal length / landscape just fine, I don't need my Z8 for that. But a phone doesn't get anywhere close to what I can do with a 300mm or longer focal length lens on the Z8 with fast moving subjects.

I started with a second-hand Canon 20D back in like 2004 or something, only upgraded when I got a deal on an old 7D and only recently bought a new R6II and the autofocus is NIGHT AND DAY

I started buying the EF mount superfast primes because they're affordable now, but the 7D (more likely it was me) couldn't get the focus just right with such a shallow DOF

The R6 just doesn't miss. Low light/high ISO image quality is also MILES better.

Cameras are not in a death spiral. Artistically speaking, phones can't do what even a low end slr/mirrorless can do, its just that phones are good enough for the low-effort content 95% of people are interested in producing. Standalone cameras are inconvenient, bulky and require some level of artistic intention.

>Does any know how much volume there would be if cameras could be used in manufacturing processes for machine vision, on robots / drones, in self-driving cars, on building for security, as webcams for video conferencing, for remote education, and everywhere else imaging is exploding?

I don't know about the manufacturing or drone stuff, but for video conferencing and remote education, the point of the video really isn't image quality or "art" but just good enough picture to not get in the way of the real purpose of the interaction, so a whole camera kit is just added complexity/annoyance for no benefit.

IMO

Depends. When a new camera is out, it often takes time before Lightroom, Capture One, etc. support it. While I don't care about how the raw format works, being able to keep using my usual software even with a brand new camera is something I care a lot about.

I thought the main value of this article was that they went out and asked various vendors that question ("why proprietary formats?") and put all their answers in one place. Too bad Nikon and Fujifilm didn't respond, but I can imagine their motivations being similar to other vendors.

How? Support for RAW formats is reasonably complete. I can hop between different editors without much, if any, hassle at all. Since getting my first DSLR in the early 00's, I have used (in no order) Photos, Bibble, Lightroom, Aperture, Capture One, Photoshop, Pixelmator, Photomator, Darkroom, On1, Raw Power, Nitro Photo, Luminar, Darktable and RawTherapee, all without fuss. Where is the lock in?

The company literally makes the hardware... You choose which company you prefer and generally stick within that ecosystem. Almost everybody uses third party software to process the images anyway, so we aren't really "locked in"

If anything maybe you'd have a point if you said they should open up the specs of the mount and lens/body communication, but the RAW format really just has near-zero impact in the real world