Raw size: another d800 firmware quirk

[paulraphael]

I can't be the only one here who uses 14 bit raw with lossless compreesion.

Nikon accurately estimates size of these files as 41MB, down from the 74MB of the uncompressed version.

But on page 436 of the manual, they say, oddly, that you can fit the same number of compressed files as uncompressed on an 8gb card: 103 images. My version of the math says the answer should be more like 185.

So what, a typo. But my camera's firmware makes the same mistake! I put in a 16GB card and the camera estimates I have room for 200 exposures. There's actually room for close to 400. As I fill up the card, the estimates stay weird ... the camera knows exactly how much space is left, but keeps basing its estimate on the assumption that my files will be 74mb.

Anyone else noticed this? Hard to believe this made it past the first round of firmware tweaks.

How do you report something like this to Nikon? Or does that question just inspire laughter?

 

[Jack]

I only shoot 14-bit uncompressed, and I buy BIG cards, so it's a non-issue for me. That said, I *suspect* you'll end up actually getting more compressed images on your card than the camera estimates it will hold.

 

[Guy Mancuso]

I get 400 images on a 32gb uncompressed 14bit

 

[paulraphael]

... I *suspect* you'll end up actually getting more compressed images on your card than the camera estimates it will hold.

Oh, absolutely. I get as many as the math would predict. I'm just noticing that the camera's number is based on mistaken information. So it's always wrong by close to a factor of two.

 

[jlancasterd]

I get 400 images on a 32gb uncompressed 14bit

Me too…

The only problem I foresee is that the hard disc on my iMAC will fill up fairly rapidly unless I routinely delete the 'spare' images that don't quite come up to standard… Discipline Dobson. Discipline…!

 

[Jack]

Me too…

The only problem I foresee is that the hard disc on my iMAC will fill up fairly rapidly unless I routinely delete the 'spare' images that don't quite come up to standard… Discipline Dobson. Discipline…!

It's called a RAID array

 

[etrigan63]

This is nothing new. My venerable D700 routinely underestimates number of shots left on the card. There are times when I shoot that the number does not decrease after the shot. Obviously calculated using "Microsoft minutes" algorithm (where the estimated time left in a software install has no actual relation to real time).

 

[gustavo]

I only shoot 14-bit uncompressed, and I buy BIG cards, so it's a non-issue for me. That said, I *suspect* you'll end up actually getting more compressed images on your card than the camera estimates it will hold.

Jack, may you share which the reason you use 14-bit uncompressed and no 14-bit lossless compressed is?

Gustavo.

 

[gustavo]

Jack, may you share which the reason you use 14-bit uncompressed and no 14-bit lossless compressed is?

Gustavo.

which one is the reason to use 14-bit uncompressed and no 14-bit lossless compressed?
regards.

 

[eleanorbrown]

Has anyone done any exact testing comparing the RAW lossless compressed with the uncompressed? Thanks, Eleanor

 

[gustavo]

Has anyone done any exact testing comparing the RAW lossless compressed with the uncompressed? Thanks, Eleanor

not me, but I´m very interesting in this issue. thank´s Eleanor.

 

[ausemmao]

I can't be the only one here who uses 14 bit raw with lossless compreesion.

Nikon accurately estimates size of these files as 41MB, down from the 74MB of the uncompressed version.

But on page 436 of the manual, they say, oddly, that you can fit the same number of compressed files as uncompressed on an 8gb card: 103 images. My version of the math says the answer should be more like 185.

So what, a typo. But my camera's firmware makes the same mistake! I put in a 16GB card and the camera estimates I have room for 200 exposures. There's actually room for close to 400. As I fill up the card, the estimates stay weird ... the camera knows exactly how much space is left, but keeps basing its estimate on the assumption that my files will be 74mb.

Anyone else noticed this? Hard to believe this made it past the first round of firmware tweaks.

How do you report something like this to Nikon? Or does that question just inspire laughter?

It's like this on every Nikon. It's a conservative estimate. While 41MB is the typical expected size, there are images that could hypothetically not change size after lossless compression, or change very little. So Nikon set it up such that the number you see is the minimum guaranteed. It then looks at the remaining space and recalculates after you take a shot. It's better than them showing a number and you suddenly coming up empty, right?

Has anyone done any exact testing comparing the RAW lossless compressed with the uncompressed? Thanks, Eleanor

There is literally no difference in the data. You could reconstruct the uncompressed file bit for bit from the losslessly compressed file. Does exactly what it says on the tin.

 

[Bob]

I have compared Nikon lossless compression with uncompressed, and as far as I can tell there is NO difference. Looking into the matter more deeply, it seems that there are two sorts of compression available in Nikon files depending of the camera. Compressed and lossless-compressed. Both forms of compression are identical as far as the image data is concerned. It is a algorithm similar to zip which returns the same bits as went it.
The difference between compressed and lossless compressed is that in the compressed version, the files is tone-mapped in much the same way that M8 files were. The far ends of the curve, both shadow and highlights effectively were encoded with fewer bits as folks tend not be be able to see a one bit difference in these parts of the tone curve. In lossless compression, that mapping does not occur.
There may be differences in other issues regarding processing time to un-compress these files, which may be tour to a degree depending on the speed of your computer implementation. I find that end to be insignificant for my purposes. The other part may be related to the time it takes to write data to the card. There it depends on the speed of the card where some lower speed cards may actually write faster with a compressed format. There is also some thought that the in-camera image buffer MAY hold more compressed files. I am not quick enough on the stopwatch or trigger button to observe this.
If I had only the compressed option available, then I would use uncompressed, but I am comfortable in using their lossless-compressed format all the time.
-bob

 

[eleanorbrown]

Ok, now assuming there is absolutely no difference in the data...why would anyone bother to shoot only uncompressed?? Guy..Jack??? I'm just interested. I have been shooting 14 bit lossless compressed all along as it never occurred to me to use up extra card and hard drive space with uncompressed. Eleanor



There is literally no difference in the data. You could reconstruct the uncompressed file bit for bit from the losslessly compressed file. Does exactly what it says on the tin.[/QUOTE]

 

[Bob]

It's like this on every Nikon. It's a conservative estimate. While 41MB is the typical expected size, there are images that could hypothetically not change size after lossless compression, or change very little. So Nikon set it up such that the number you see is the minimum guaranteed. It then looks at the remaining space and recalculates after you take a shot. It's better than them showing a number and you suddenly coming up empty, right?




There is literally no difference in the data. You could reconstruct the uncompressed file bit for bit from the losslessly compressed file. Does exactly what it says on the tin.

Note that the degree of compression that can be achieved by the Nikon algorithm depends on the amount of detail in the file. The compressed file can be about half an uncompressed file up to slightly bigger depending on the amount of high frequency detail and noise.
The poor little camera brain just has no idea how much detail that you will be shooting in the future so it just uses the full size file in its shots remaining estimate. Better to have gas in the tank when the needle hits zero than nothing when it reads 1/8 LOL
-bob

 

[Bob]

Ok, now assuming there is absolutely no difference in the data...why would anyone bother to shoot only uncompressed?? Guy..Jack??? I'm just interested. I have been shooting 14 bit lossless compressed all along as it never occurred to me to use up extra card and hard drive space with uncompressed. Eleanor



There is literally no difference in the data. You could reconstruct the uncompressed file bit for bit from the losslessly compressed file. Does exactly what it says on the tin.

[/QUOTE]


Well there is some degree of traditional suspicion of any sort of compression, and there is a potential for differences in processing time both in-camera+card writing time as well as post processing of NEFS. If you tend to convert large numbers of files you may notice the difference, but if you tend to convert one at a time for further processing, then it may not be important to you.

I personally think that this is one of those issues that cause endless discussion precisely because there is no clear winning case for one vs the other.
-bob

 

[ausemmao]

Ok, now assuming there is absolutely no difference in the data...why would anyone bother to shoot only uncompressed?? Guy..Jack??? I'm just interested. I have been shooting 14 bit lossless compressed all along as it never occurred to me to use up extra card and hard drive space with uncompressed. Eleanor

I couldn't say for certain, but I'd imagine inertia+uncertainty. The D1 didn't have lossless compression, so uncompressed was the only option, and probably more importantly, as Bob says above, there's often a mistrust of anything that has 'compression' in its name (even though no other camera makers offer an uncompressed RAW) if people don't understand the underlying methods. Nikon doesn't exactly help things with their bad naming of the different modes and the lack of documentation. With modern processors disk performance is more of a bottleneck than CPU for file opening so even that aspect isn't much of an advantage anymore.

Aside: if you want to try something interesting, compare lossy compression to lossless with even heavy PP, and see if you can tell the difference between them. I tried it, and now shoot 14 bit compressed. I could not find any real differences. 14 bit vs 12 bit I could see differences after post processing, but only in darker areas of the image.

 

[Jan Brittenson]

There are some differences in encoding in lossless vs uncompressed. Lossless compressed NEFs are stored in a camera specific format, and the best argument for uncompressed is software compatibility for raw processors that can't read the D800 lossless compressed files correctly.

Personally I shoot lossless compressed. It makes the buffer last longer and speeds up both writes to the card and importing on the computer. I don't really use lossy compressed, but wouldn't hesitate to use it if I needed to shrink the files further.

 

[paulraphael]

I have compared Nikon lossless compression with uncompressed, and as far as I can tell there is NO difference.

Yup. Lossless compression algorithms have been around a long, long time. Zip, RLE, and LZW are examples. An uncompressed file is bit-for-bit identical to the original. If this were not the case, you couldn't call it lossless.

The maximum degree of compression is considerably less impressive than what's possible with lossy compression. Unsurprisingly.

In both types, the degree of compression possible depends upon the quantity of actual information in the file. Photographs with big undetailed areas, music with lots of silence, etc., compress a lot.

In blind side-by-side tests, people have a difficult to impossible time discerning higher quality versions of lossy compression from uncompressed originals. But a lot of us still prefer lossless compression on general principle.

 

[Bryan Stephens]

I had noticed this on my D3s and thought it was my camera, but someone explained that the camera does the initial computations based on uncompressed for remaining images on a card, even though you have switched the camera to lossless compressed.

I found that the more images I took on a card, the remaining captures would actually sometimes go up to compensate.