But it seems that if you are re-encoding the image anyways it is best to just generate it in the "correct" orientation and skip this metadata. Emitting the metadata seems mostly useful if you are trying to avoid re-encoding the image.

Or is there a benefit to generating the thumbnail in the original rotation then preserving the metadata?

Is jpeg the only format which can be “rotated losslessly” in this way, ie without exif metadata (and excluding any obvious formats like bitmaps or whatever can be trivially rotated) ?

Hello, libvips author here, the JPEG loader has a flag to do this for you. You can write:

VipsImage *image = vips_image_new_from_file("something.jpg", "autorotate", TRUE, NULL);

and it'll flip it upright for you and remove any EXIF orientation tag. It isn't the default since it has (obviously) a large memory and cpu cost.

For simple resize and crop it's much better to use the exif tag to adjust the resize and crop parameters (the vipsthumbnail command does this).

One reason I can think of: when you rotate an image in the windows photo viewer (and probably lots of other apps), it stores that in the EXIF metadata. So the rotation is lossless, by not having to re-compress the JPEG.

I could also imagine that the earliest digital cameras wouldn't have had the processing power, or RAM to store the entire image in memory after reading it from the sensor, in order to do such a rotation. Hence EXIF rotation as a cheap alternative.

You need to appreciate how little RAM and CPU power those cameras have.

As far as cost:

Applying the right tag: 0

Anything else: not 0

Why not include a faster CPU or more RAM? In an industry where people pick one product over the other because of a $10 price difference, you’re never going to convince management to approve the better CPU when a tag solves the problem.

These are hardware companies. They look at software as a nuisance. “fix it in software” is for the downstream.

There are sort of two questions here:

> why does EXIF transform metadata exist?

Because there are times when it is useful to be able to quickly and losslessly rotate a photo. For example an image viewer might provide quick options to rotate a photo (in case it did end up rotated the wrong way originally).

Probably this should just have been part of the actual image format rather than metadata, but that isn't how it played out.

> why doesn't the camera itself perform a simple rotation or mirror

There could be various reasons. The simplest is that adjusting the rotation naturally happens later in the pipeline than lossy encoding. You don't want to re-encode for quality and performance reasons so just slap on the metadata. The other could be that it is less expensive to avoid rotating. In many cameras the encoding is handled by specialized hardware, adding a way to rotate the raw pixels is likely more expensive than always having the source pixel map to the same input pixel in the encoder. There is no hard reason it couldn't be done, but in many situations it is cheaper to do it this way.

I think it does this after the photo has already been stored as JPEG. I think users would expect the image be persisted the moment you capture it, so any transformations after that are better done in metadata. When the image has been saved as JPEG the original image data is discarded so you can't transform without deteriorating the image.

I don't know why I should have a preference when it comes to the memory layout of an image. If metadata tells be to orient it this way, I'll orient it this way, if that way, then that way. It seems weird and wasteful to me (and potentially prone to error) to want to reorder the data in order to change the display orientation.

I can only imagine caring when it comes to streaming a very large image serially, and then I still have no idea what preference I would have (other than that I get the metadata first.)

As an analogy, if I have some irreplaceable video that has a bad aspect ratio, I fix it in the metadata, I don't transcode it. The data is a blob, the metadata is what tells me how to display it (even if it's just the mimetype.)

I too have always found this strange but I have never found a solid reason. Having an explicit declaration of intent is not a terrible feature in a data type so this is not exactly a bad choice. Also this means that a camera roll from a single sensor can be stored as an array in C since every image has the exact same size. It also does make life easier for the camera developers (but harder for photo viewer developers). If anyone knows the history here I would actually really be interested.

Applying rotation can have a large performance penalty (both cpu and memory), so you want to be able to avoid it if possible.

Probably for a higher-level crate, (like the one the author wrote), but it adds computational complexity, and you don't want that by default with a low level crate like `image`.

For example, if you wanted to blur or grayscale an image, do you really want to rotate it? When you re-save it, you'll have to remove the exif rotation metadata too.

If it's all in rust you can maybe probably compile it to wasm with out much effort. Kinda depends what libraries they used / how they factored their code etc.