Site Sponsors
Results 1 to 21 of 21

Thread: Raw size: another d800 firmware quirk

  1. #1
    Senior Member
    Join Date
    Mar 2012
    Location
    NYC
    Posts
    333
    Post Thanks / Like

    Raw size: another d800 firmware quirk

    I can't be the only one here who uses 14 bit raw with lossless compreesion.

    Nikon accurately estimates size of these files as 41MB, down from the 74MB of the uncompressed version.

    But on page 436 of the manual, they say, oddly, that you can fit the same number of compressed files as uncompressed on an 8gb card: 103 images. My version of the math says the answer should be more like 185.

    So what, a typo. But my camera's firmware makes the same mistake! I put in a 16GB card and the camera estimates I have room for 200 exposures. There's actually room for close to 400. As I fill up the card, the estimates stay weird ... the camera knows exactly how much space is left, but keeps basing its estimate on the assumption that my files will be 74mb.

    Anyone else noticed this? Hard to believe this made it past the first round of firmware tweaks.

    How do you report something like this to Nikon? Or does that question just inspire laughter?

  2. #2
    Sr. Administrator Jack's Avatar
    Join Date
    Oct 2007
    Location
    Los Altos, CA
    Posts
    10,486
    Post Thanks / Like
    Images
    1031

    Re: Raw size: another d800 firmware quirk

    I only shoot 14-bit uncompressed, and I buy BIG cards, so it's a non-issue for me. That said, I *suspect* you'll end up actually getting more compressed images on your card than the camera estimates it will hold.
    Jack
    home: www.getdpi.com

    "Perfection is not attainable. But if we chase perfection, we can catch excellence."

  3. #3
    Administrator, Instructor Guy Mancuso's Avatar
    Join Date
    Oct 2007
    Location
    Arizona
    Posts
    23,623
    Post Thanks / Like
    Images
    2555

    Re: Raw size: another d800 firmware quirk

    I get 400 images on a 32gb uncompressed 14bit
    Photography is all about experimentation and without it you will never learn art.

    www.guymancusophotography.com

  4. #4
    Senior Member
    Join Date
    Mar 2012
    Location
    NYC
    Posts
    333
    Post Thanks / Like

    Re: Raw size: another d800 firmware quirk

    Quote Originally Posted by Jack View Post
    ... I *suspect* you'll end up actually getting more compressed images on your card than the camera estimates it will hold.
    Oh, absolutely. I get as many as the math would predict. I'm just noticing that the camera's number is based on mistaken information. So it's always wrong by close to a factor of two.

  5. #5
    Member jlancasterd's Avatar
    Join Date
    Jul 2009
    Location
    Garndolbenmaen, Wales
    Posts
    164
    Post Thanks / Like

    Re: Raw size: another d800 firmware quirk

    Quote Originally Posted by Guy Mancuso View Post
    I get 400 images on a 32gb uncompressed 14bit
    Me too…

    The only problem I foresee is that the hard disc on my iMAC will fill up fairly rapidly unless I routinely delete the 'spare' images that don't quite come up to standard… Discipline Dobson. Discipline…!
    John L Dobson
    Editor, Ffestiniog Railway Magazine

  6. #6
    Sr. Administrator Jack's Avatar
    Join Date
    Oct 2007
    Location
    Los Altos, CA
    Posts
    10,486
    Post Thanks / Like
    Images
    1031

    Re: Raw size: another d800 firmware quirk

    Quote Originally Posted by jlancasterd View Post
    Me too…

    The only problem I foresee is that the hard disc on my iMAC will fill up fairly rapidly unless I routinely delete the 'spare' images that don't quite come up to standard… Discipline Dobson. Discipline…!
    It's called a RAID array
    Jack
    home: www.getdpi.com

    "Perfection is not attainable. But if we chase perfection, we can catch excellence."

  7. #7
    Super Duper
    Senior Member
    etrigan63's Avatar
    Join Date
    Nov 2007
    Location
    Earth, Sol System (near Miami, FL)
    Posts
    2,501
    Post Thanks / Like
    Images
    21

    Re: Raw size: another d800 firmware quirk

    This is nothing new. My venerable D700 routinely underestimates number of shots left on the card. There are times when I shoot that the number does not decrease after the shot. Obviously calculated using "Microsoft minutes" algorithm (where the estimated time left in a software install has no actual relation to real time).
    Carlos Echenique | Carlos Echenique Photography |Olympus OM-D E-M1 MK II | Olympus Pen-F - M.Zuiko 17mm f/1.8, M.Zuiko 25mm f/1.8, M.Zuiko 45mm f/1.8, Rokinon 12mm f/2 NCS, M.Zuiko 75mm f/1.8, M.Zuiko 12-40mm f/2.8 PRO, M.Zuiko 40-150mm f/2.8 PRO

  8. #8
    Member
    Join Date
    Apr 2012
    Location
    Buenos Aires, Argentina
    Posts
    67
    Post Thanks / Like
    Images
    3

    Re: Raw size: another d800 firmware quirk

    Quote Originally Posted by Jack View Post
    I only shoot 14-bit uncompressed, and I buy BIG cards, so it's a non-issue for me. That said, I *suspect* you'll end up actually getting more compressed images on your card than the camera estimates it will hold.
    Jack, may you share which the reason you use 14-bit uncompressed and no 14-bit lossless compressed is?

    Gustavo.

  9. #9
    Member
    Join Date
    Apr 2012
    Location
    Buenos Aires, Argentina
    Posts
    67
    Post Thanks / Like
    Images
    3

    Re: Raw size: another d800 firmware quirk

    Quote Originally Posted by gustavo View Post
    Jack, may you share which the reason you use 14-bit uncompressed and no 14-bit lossless compressed is?

    Gustavo.
    which one is the reason to use 14-bit uncompressed and no 14-bit lossless compressed?
    regards.

  10. #10
    Senior Member eleanorbrown's Avatar
    Join Date
    Feb 2009
    Location
    Houston and Keystone, Colorado
    Posts
    706
    Post Thanks / Like
    Images
    40

    Re: Raw size: another d800 firmware quirk

    Has anyone done any exact testing comparing the RAW lossless compressed with the uncompressed? Thanks, Eleanor

  11. #11
    Member
    Join Date
    Apr 2012
    Location
    Buenos Aires, Argentina
    Posts
    67
    Post Thanks / Like
    Images
    3

    Re: Raw size: another d800 firmware quirk

    Quote Originally Posted by eleanorbrown View Post
    Has anyone done any exact testing comparing the RAW lossless compressed with the uncompressed? Thanks, Eleanor
    not me, but I´m very interesting in this issue. thank´s Eleanor.

  12. #12
    Member
    Join Date
    Jun 2012
    Posts
    44
    Post Thanks / Like

    Re: Raw size: another d800 firmware quirk

    Quote Originally Posted by paulraphael View Post
    I can't be the only one here who uses 14 bit raw with lossless compreesion.

    Nikon accurately estimates size of these files as 41MB, down from the 74MB of the uncompressed version.

    But on page 436 of the manual, they say, oddly, that you can fit the same number of compressed files as uncompressed on an 8gb card: 103 images. My version of the math says the answer should be more like 185.

    So what, a typo. But my camera's firmware makes the same mistake! I put in a 16GB card and the camera estimates I have room for 200 exposures. There's actually room for close to 400. As I fill up the card, the estimates stay weird ... the camera knows exactly how much space is left, but keeps basing its estimate on the assumption that my files will be 74mb.

    Anyone else noticed this? Hard to believe this made it past the first round of firmware tweaks.

    How do you report something like this to Nikon? Or does that question just inspire laughter?
    It's like this on every Nikon. It's a conservative estimate. While 41MB is the typical expected size, there are images that could hypothetically not change size after lossless compression, or change very little. So Nikon set it up such that the number you see is the minimum guaranteed. It then looks at the remaining space and recalculates after you take a shot. It's better than them showing a number and you suddenly coming up empty, right?


    Quote Originally Posted by eleanorbrown View Post
    Has anyone done any exact testing comparing the RAW lossless compressed with the uncompressed? Thanks, Eleanor
    There is literally no difference in the data. You could reconstruct the uncompressed file bit for bit from the losslessly compressed file. Does exactly what it says on the tin.

  13. #13
    Administrator Bob's Avatar
    Join Date
    Oct 2007
    Location
    Prescott, Arizona
    Posts
    4,492
    Post Thanks / Like
    Images
    367

    Re: Raw size: another d800 firmware quirk

    I have compared Nikon lossless compression with uncompressed, and as far as I can tell there is NO difference. Looking into the matter more deeply, it seems that there are two sorts of compression available in Nikon files depending of the camera. Compressed and lossless-compressed. Both forms of compression are identical as far as the image data is concerned. It is a algorithm similar to zip which returns the same bits as went it.
    The difference between compressed and lossless compressed is that in the compressed version, the files is tone-mapped in much the same way that M8 files were. The far ends of the curve, both shadow and highlights effectively were encoded with fewer bits as folks tend not be be able to see a one bit difference in these parts of the tone curve. In lossless compression, that mapping does not occur.
    There may be differences in other issues regarding processing time to un-compress these files, which may be tour to a degree depending on the speed of your computer implementation. I find that end to be insignificant for my purposes. The other part may be related to the time it takes to write data to the card. There it depends on the speed of the card where some lower speed cards may actually write faster with a compressed format. There is also some thought that the in-camera image buffer MAY hold more compressed files. I am not quick enough on the stopwatch or trigger button to observe this.
    If I had only the compressed option available, then I would use uncompressed, but I am comfortable in using their lossless-compressed format all the time.
    -bob

  14. #14
    Senior Member eleanorbrown's Avatar
    Join Date
    Feb 2009
    Location
    Houston and Keystone, Colorado
    Posts
    706
    Post Thanks / Like
    Images
    40

    Re: Raw size: another d800 firmware quirk

    Ok, now assuming there is absolutely no difference in the data...why would anyone bother to shoot only uncompressed?? Guy..Jack??? I'm just interested. I have been shooting 14 bit lossless compressed all along as it never occurred to me to use up extra card and hard drive space with uncompressed. Eleanor



    There is literally no difference in the data. You could reconstruct the uncompressed file bit for bit from the losslessly compressed file. Does exactly what it says on the tin.[/QUOTE]

  15. #15
    Administrator Bob's Avatar
    Join Date
    Oct 2007
    Location
    Prescott, Arizona
    Posts
    4,492
    Post Thanks / Like
    Images
    367

    Re: Raw size: another d800 firmware quirk

    Quote Originally Posted by ausemmao View Post
    It's like this on every Nikon. It's a conservative estimate. While 41MB is the typical expected size, there are images that could hypothetically not change size after lossless compression, or change very little. So Nikon set it up such that the number you see is the minimum guaranteed. It then looks at the remaining space and recalculates after you take a shot. It's better than them showing a number and you suddenly coming up empty, right?




    There is literally no difference in the data. You could reconstruct the uncompressed file bit for bit from the losslessly compressed file. Does exactly what it says on the tin.

    Note that the degree of compression that can be achieved by the Nikon algorithm depends on the amount of detail in the file. The compressed file can be about half an uncompressed file up to slightly bigger depending on the amount of high frequency detail and noise.
    The poor little camera brain just has no idea how much detail that you will be shooting in the future so it just uses the full size file in its shots remaining estimate. Better to have gas in the tank when the needle hits zero than nothing when it reads 1/8 LOL
    -bob

  16. #16
    Administrator Bob's Avatar
    Join Date
    Oct 2007
    Location
    Prescott, Arizona
    Posts
    4,492
    Post Thanks / Like
    Images
    367

    Re: Raw size: another d800 firmware quirk

    Quote Originally Posted by eleanorbrown View Post
    Ok, now assuming there is absolutely no difference in the data...why would anyone bother to shoot only uncompressed?? Guy..Jack??? I'm just interested. I have been shooting 14 bit lossless compressed all along as it never occurred to me to use up extra card and hard drive space with uncompressed. Eleanor



    There is literally no difference in the data. You could reconstruct the uncompressed file bit for bit from the losslessly compressed file. Does exactly what it says on the tin.
    [/QUOTE]


    Well there is some degree of traditional suspicion of any sort of compression, and there is a potential for differences in processing time both in-camera+card writing time as well as post processing of NEFS. If you tend to convert large numbers of files you may notice the difference, but if you tend to convert one at a time for further processing, then it may not be important to you.

    I personally think that this is one of those issues that cause endless discussion precisely because there is no clear winning case for one vs the other.
    -bob
    Likes 1 Member(s) liked this post

  17. #17
    Member
    Join Date
    Jun 2012
    Posts
    44
    Post Thanks / Like

    Re: Raw size: another d800 firmware quirk

    Quote Originally Posted by eleanorbrown View Post
    Ok, now assuming there is absolutely no difference in the data...why would anyone bother to shoot only uncompressed?? Guy..Jack??? I'm just interested. I have been shooting 14 bit lossless compressed all along as it never occurred to me to use up extra card and hard drive space with uncompressed. Eleanor
    I couldn't say for certain, but I'd imagine inertia+uncertainty. The D1 didn't have lossless compression, so uncompressed was the only option, and probably more importantly, as Bob says above, there's often a mistrust of anything that has 'compression' in its name (even though no other camera makers offer an uncompressed RAW) if people don't understand the underlying methods. Nikon doesn't exactly help things with their bad naming of the different modes and the lack of documentation. With modern processors disk performance is more of a bottleneck than CPU for file opening so even that aspect isn't much of an advantage anymore.

    Aside: if you want to try something interesting, compare lossy compression to lossless with even heavy PP, and see if you can tell the difference between them. I tried it, and now shoot 14 bit compressed. I could not find any real differences. 14 bit vs 12 bit I could see differences after post processing, but only in darker areas of the image.

  18. #18
    Senior Subscriber Member
    Join Date
    Nov 2007
    Location
    San Francisco, CA
    Posts
    1,306
    Post Thanks / Like

    Re: Raw size: another d800 firmware quirk

    There are some differences in encoding in lossless vs uncompressed. Lossless compressed NEFs are stored in a camera specific format, and the best argument for uncompressed is software compatibility for raw processors that can't read the D800 lossless compressed files correctly.

    Personally I shoot lossless compressed. It makes the buffer last longer and speeds up both writes to the card and importing on the computer. I don't really use lossy compressed, but wouldn't hesitate to use it if I needed to shrink the files further.

  19. #19
    Senior Member
    Join Date
    Mar 2012
    Location
    NYC
    Posts
    333
    Post Thanks / Like

    Re: Raw size: another d800 firmware quirk

    Quote Originally Posted by Bob View Post
    I have compared Nikon lossless compression with uncompressed, and as far as I can tell there is NO difference.
    Yup. Lossless compression algorithms have been around a long, long time. Zip, RLE, and LZW are examples. An uncompressed file is bit-for-bit identical to the original. If this were not the case, you couldn't call it lossless.

    The maximum degree of compression is considerably less impressive than what's possible with lossy compression. Unsurprisingly.

    In both types, the degree of compression possible depends upon the quantity of actual information in the file. Photographs with big undetailed areas, music with lots of silence, etc., compress a lot.

    In blind side-by-side tests, people have a difficult to impossible time discerning higher quality versions of lossy compression from uncompressed originals. But a lot of us still prefer lossless compression on general principle.

  20. #20
    Workshop Member Bryan Stephens's Avatar
    Join Date
    Oct 2011
    Location
    Connecticut
    Posts
    463
    Post Thanks / Like

    Re: Raw size: another d800 firmware quirk

    I had noticed this on my D3s and thought it was my camera, but someone explained that the camera does the initial computations based on uncompressed for remaining images on a card, even though you have switched the camera to lossless compressed.

    I found that the more images I took on a card, the remaining captures would actually sometimes go up to compensate.
    Bryan

    “You don’t take a photograph, you make it.” — Ansel Adams

  21. #21
    Senior Member
    Join Date
    Mar 2012
    Location
    NYC
    Posts
    333
    Post Thanks / Like

    Re: Raw size: another d800 firmware quirk

    Quote Originally Posted by Bryan Stephens View Post
    I had noticed this on my D3s and thought it was my camera, but someone explained that the camera does the initial computations based on uncompressed for remaining images on a card, even though you have switched the camera to lossless compressed.
    Yeah, it's clear that this is what's happening. It's just dumb. It would take half a minute for some engineer to write the code properly.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •