Site Sponsors
Results 1 to 29 of 29

Thread: Unlock 16-bit RAW files for M8

  1. #1
    Member
    Join Date
    Nov 2009
    Posts
    74
    Post Thanks / Like

    Unlock 16-bit RAW files for M8

    M8 is my poor man's MM and I am very glad to stumble into this discovery via this thread http://www.reddotforum.com/showthrea...e-M8-m8raw2dng

    Basically, the sensor data has 16 bit but Leica firmware only saves in 8 bit, throwing away info in the shadow.

    Arvid discovered how to set M8 to output 16-bit in Leica's RAW format and wrote a tool to convert the output to DNG. It works for C1, LR, PS Raw Editor. If you use C1, don't use option 'c' when converting RAW to DNG. Very cool.

    M8raw2dng – Tool to convert Leica M8 RAW files to DNG


    --Cuong
    Likes 1 Member(s) liked this post

  2. #2
    Senior Member
    Join Date
    Dec 2007
    Location
    Jerusalem, Israel
    Posts
    1,282
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Quote Originally Posted by cuonghuutran View Post
    I am very glad to stumble into this discovery via this thread "RAW" uncompressed format unlocked for the M8: m8raw2dng

    Basically, the sensor data has 16 bit but Leica firmware only saves in 8 bit, throwing away info in the shadow.

    Arvid discovered how to set M8 to output 16-bit in Leica's RAW format and wrote a tool to convert the output to DNG. It works for C1, LR, PS Raw Editor. If you use C1, don't use option 'c' when converting RAW to DNG. Very cool.

    M8raw2dng – Tool to convert Leica M8 RAW files to DNG


    --Cuong
    This is deep in geek territory, and so far only applies to the M8, but I'm intrigued... What these folks have discovered is the "Easter egg" procedure to launch a set of service routines on an M8. And one side effect of the M8 service routine is that it can set the camera to skip the compression step of taking the square root of the intensity signal for each pixel and output the full 12-14 bits of usable data (in a 16-bit field, I presume) as an undocumented RAW file. The site these links take you to also offers software to turn the RAW file back into a useable DNG, without the loss of any precision. I guess they make the files look as if they came from an M9.

    Here's how to do it:

    If you want to set your M8 to record the RAW files you have to follow the following Steps.

    To activate the service mode press

    4 times right arrow button
    3 times left arrow button
    1 time right arrow button
    INFO button
    Ignore the stuff that appears on screen
    Touch the shutter lightly to exit the service menu

    Now you can press SET and the compression settings will show an additional option JPG fine+RAW at the bottom of the list. Select it and you’re done.

    caveat: the setting will be deactivated on power-off (manual or automatic energy save mode) On next reboot the compression will be set to JPG fine only. Beware of forgetting to change it otherwise you’ll be shooting JPG only… .....
    Some things to add -- I wouldn't ignore the stuff that appears on the screen -- it is a series of simple and useful tests, some of which might be done in a store to see if service is needed. They show every external control input as received in the camera, for example, the three degrees of shutter press. If a lens is attached the lens code shows, resolving the 6-bit plus 2 bits of frame preview to an 8-bit number (between 0 and 256), as some of us had long suspected. That part seems useful for helping Sharpie-coders get their lines right. You exit each test by pressing SET, but you can try a different one by pressing INFO again. Power OFF and then ON gets you back to normal operation.

    I have one M9 which sometimes doesn't do a shutter half-press correctly, and this sort of test would help me nail down the issue. But the M9 "Easter egg" is different, and not known yet. Google hasn't picked it up. Does anyone know how to do this for the later models, and care to share the information???

    scott

  3. #3
    Senior Member JohnBrew's Avatar
    Join Date
    Jun 2013
    Location
    South Carolina
    Posts
    532
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Not sure how to get this to work. Is it imported directly into PS?

  4. #4
    Senior Member
    Join Date
    Dec 2007
    Location
    Jerusalem, Israel
    Posts
    1,282
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    I haven't tried the 16bit M8 files and their conversion into dng yet. My understanding from what I read is that you shoot after using this trick to set it up, and get L10xxxx.RAW fiiles, and the downloaded program converts these into M9-like .dng files, which any software should be able to render into .jpg or .tif.

    scott

  5. #5
    Senior Member MaxKißler's Avatar
    Join Date
    Nov 2010
    Location
    Berlin
    Posts
    387
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Quote Originally Posted by scott kirkpatrick View Post
    ...
    I have one M9 which sometimes doesn't do a shutter half-press correctly, and this sort of test would help me nail down the issue. But the M9 "Easter egg" is different, and not known yet. Google hasn't picked it up. Does anyone know how to do this for the later models, and care to share the information???

    scott
    I'd like to know that too...

  6. #6
    Member lenticularis's Avatar
    Join Date
    Apr 2012
    Location
    Canberra, Australia
    Posts
    41
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Quote Originally Posted by scott kirkpatrick View Post
    I haven't tried the 16bit M8 files and their conversion into dng yet. My understanding from what I read is that you shoot after using this trick to set it up, and get L10xxxx.RAW fiiles, and the downloaded program converts these into M9-like .dng files, which any software should be able to render into .jpg or .tif.

    scott

    Yes that's it basically. It stores a 16bitRAW file and a matching jpeg. Run the author's software and it will create a DNG file that is then editable in your software of choice. I use it all the time now with my M8...

    The author is also extremely responsive to queries. I found that it would produce files but they were not previewable in Capture One Pro 7 (my RAW/DNG editor of choice for any camera). Using the -p option when creating the files solved the issue for me. See the author's website for description of the command line options.

  7. #7
    Member
    Join Date
    Nov 2009
    Posts
    74
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    These two images compare the dynamic range between compressed vs. uncompressed RAW on M8. Using the script downloaded from the site, I can't open it with C1 Pro 7. I asked the author and he recommended me to remove the "-c" option and then it works fine with C1.



    Likes 4 Member(s) liked this post

  8. #8
    Subscriber Member jaapv's Avatar
    Join Date
    Nov 2007
    Posts
    770
    Post Thanks / Like
    Images
    250

    Re: Unlock 16-bit RAW files for M8

    It still begs the question why Leica decided to disable the uncompressed option on the M8 originally. At the time it was said the camera became too slow to be useable.
    Can the bottleneck at the time have been the SD cards available?
    JAAP
    http://www.jaapvphotography.eu
    The colours of my generation are black and white.

  9. #9
    Member
    Join Date
    Feb 2009
    Posts
    176
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    I just did a very unscientific test to get an idea what the write speed difference is between the DNG and RAW+jpeg.

    I set the camera to each of the two settings and snap a few shots timing from the time I press the shutter to the time the red LED sops blinking. I am using a Transcend SDHC 16GB C10 card.

    DNG on average gives 3.6 to 3.8 seconds.

    RAW+jpeg on average gives 10.4 to 10.7 seconds.

    It could well be my response time in pressing the buttons and all those possible errors, but at least it shows quite a bit of difference in writing the files.

    Can someone else try the same thing with different cards? I don't have any faster card than this so if someone and a faster card and are getting faster speed it would be nice to know.

    Or if someone has a proper setup to do this test properly and give some data that would even be better!
    Phil

  10. #10
    Super Duper
    Senior Member
    Knorp's Avatar
    Join Date
    Dec 2007
    Posts
    4,002
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Hi Phil, perhaps you already did without mentioning it, but at least shoot "DNG+JPG fine" so it compares to the "JPG fine+RAW".

    Kind regards.
    Bart ...

  11. #11
    Senior Member
    Join Date
    Dec 2007
    Location
    Jerusalem, Israel
    Posts
    1,282
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    I used to do these tests. To answer Jaap's question, let's use the best card as recommended by Leica at the time the M8 was current. My M8 has an 8 GB SanDisk Ultra, rated at 15 MB/sec, half full. (The SanDisk Extreme cards, although rated faster, ran slower on the M8.) It is set for normal use, discreet advance and auto ISO enabled. Firmware version is 2.014 and it is an M8.U with the shutter modified and the lovely sapphire LED cover. On screen I have online stopwatch running, with a digital time output refreshed every 40 msec.

    I set the shutter to "C"ontinuous, and hold it down while focused on the onscreen time indications. For DNG only output, the M8 takes 10 shots at intervals of 0.6 sec, with the times very consistent for the first 5 shots and becoming a bit more erratic, perhaps as long as 0.7 sec on the 9 or 10th shot. Then the internal buffer fills and the time between shots slows as the camera must wait for internal buffer space to be available. The first buffer-full shot is delayed by an amount that varies from one series to the next, but the second and further buffer-full shots are separated by a consistent interval. Finally, I release the shutter and watch the red light blink, taking one more shot when it stops to record that time. The initial interval between shots (when there is plenty of buffer space) is the same whether I am shooting DNG only, DNG + fine JPEG, or RAW plus fine JPEG -- 0.6 sec. So this must be time needed to get 20 MB of unprocessed data off of the CCD. Other processing will proceed in parallel, but the camera is ready to take another shot.

    The next stage, shooting with the buffer full and needing to be drained a bit, does depend on how much data in final form will be written to the memory card. For DNG only, the intervals become 2.7 to 2.8 sec per frame. For DNG + fine JPEG, they range from 7 to 7.5 sec per frame. But for RAW plus fine jpeg they are only 5.5 sec, or about twice the time required to clear buffer space for a compressed DNG file.

    The final interval also depended on the type of data being written. To clear the entire buffer and stop the red light blinking took 22 sec with 8bits per pixel DNG files, 55 sec with the RAW data + fine JPEG, and 75 sec with compressed DNG + fine JPEG.

    To really understand why the RAW format wasn't used we still have to understand what the actual parallel processes are that are performed after the data is in the buffer and whether the camera only writes to the chip when done, performing all processing in the internal buffer, or uses the chip as an intermediate store. I'm still thinking about that.

    scott

    PS: Checking the files that were written, I see that the case I thought was RAW + fine JPEG was just jpegs. I'll go back and see what I did wrong so that the RAW selection didn't take hold. So now the relative times are DNG only 22 sec to process the last shot and clear the entire buffer, fine JPEG processes the final shot and clears the entire buffer in 55 sec, and doing both (compressed DNG plus a fine JPEG) takes 75 sec to process the last shot and clear the entire buffer. Note that the time for fine JPEG + DNG is about equal to the sum of the times for each one separately. There is only one signal processing chip to do both jobs, so this is understandable.
    Last edited by scott kirkpatrick; 8th February 2014 at 09:25. Reason: the PS modifies my conclusions, requires an extra piece of work.
    Likes 1 Member(s) liked this post

  12. #12
    Member
    Join Date
    Feb 2009
    Posts
    176
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Quote Originally Posted by Knorp View Post
    Hi Phil, perhaps you already did without mentioning it, but at least shoot "DNG+JPG fine" so it compares to the "JPG fine+RAW".

    Kind regards.
    Knorp:

    Actually I have not tried that because I only shot RAW, and I was only interested in how this RAW file manuever would impact my shooting.

    However, you are right, a comparison between RAW+jpg fine and DNG+jpg fine would indicate the difference in RAW vs DNG write time better, because both are also writing JPG fine at the same time.

    So I tried again this morning and I got 8.3 to 8.5 seconds for DNG+JPG fine, which is much closer to the RAW+JPG fine time.

    I am wondering if the UHS class 3 would work on the M8 at all. It supposedly has a minimum bus speed of 30MB/s as compared to the C10's 10 MB/s.
    Phil
    Likes 1 Member(s) liked this post

  13. #13
    Senior Member
    Join Date
    Dec 2007
    Location
    Jerusalem, Israel
    Posts
    1,282
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    I see that fotism's 10 MB/sec Transcend card clears a DNG + fine JPEG in 8+ sec while my 15 MB/sec Ultra takes 7.5 sec. His DNG only takes 3.5+ sec; with my card it took 2.8 sec. Those results seem consistent with the quoted speeds of the two cards. The Extreme cards are now said to run at 30 MB/sec. I have used that line in M8s, so they work, but they were slower.

    Notice that the buffer fills at 10 shots, regardless of whether you are shooting DNG only, DNG plus JPEG, or JPEG only. That suggests that most of the interesting things are happening as the data is written to the card, not reserving extra memory in the internal buffer for the JPEG.

    scott

  14. #14
    Member
    Join Date
    Feb 2009
    Posts
    176
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    I ran some tests using the C mode and these are my numbers:

    10 shot RAW+JPG fine 1:27 to 1:29 for the red LED to stop blinking,
    10 shot DNG+JPG fine 1:08 to 1:11,
    10 shot DNG only 25 to 26s

    When shooting in RAW+JPEG fine the shots were fired with the best rhythm and at a faster pace. The blinking was long, though. The other two showed slower pace and the shot to shot intervals are a bit irregular.

    I also found that if I keep on pressing the shutter button in DNG doing C mode, it was able to squeeze in an extra shot or two, after a very brief hesitation, before the "buffer full" red box shows up. That may be due to my card being slower and there is time for the buffer to clear and make room for another shot. I was able to get a maximum of 13 shots in one go in DNG only but only 10 on the other two formats.
    Phil

  15. #15
    Senior Member
    Join Date
    Dec 2007
    Location
    Jerusalem, Israel
    Posts
    1,282
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    I rechecked my results for timing fine JPEG alone, and then got a series with RAW + fine JPEG. Again the only differences are seen once the buffer is full.

    For the first 10 shots, with shot intervals only due to writing into the buffer:
    DNG only .62 sec between shots
    JPEG only .62 sec
    DNG + JPEG .67 sec
    RAW + JPEG .55 sec (!!)
    (So if you don't fill the buffer, RAW is great!)

    Once the buffer was full, shot intervals become:
    DNG only 2.7 to 2.8 sec
    JPEG only 5.5 to 5.6 sec
    DNG + JPEG 7 to 7.5 sec
    RAW + JPEG 9.6 to 10.2 sec.
    (So the time to output a RAW is about twice the time to output a compressed DNG.)

    and time to clear a full buffer by writing out 10 exposures:
    DNG only 22 sec
    JPEG only 53 sec
    DNG + JPEG 74 sec
    RAW + JPEG 98 sec
    (again each RAW takes twice as long to output as DNG)

    scott
    Likes 2 Member(s) liked this post

  16. #16
    Senior Member dude163's Avatar
    Join Date
    Feb 2011
    Location
    New Brunswick , Canada
    Posts
    1,111
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    so how large are these RAW files? 3 x 10mp?

  17. #17
    Super Duper
    Senior Member
    Knorp's Avatar
    Join Date
    Dec 2007
    Posts
    4,002
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    So far my files are roughly 21Mb. Still having a bit of difficulty with C1P7.2 as not all converted files show up.
    Apparently, whereas Aperture seems less picky btw, the files' DNG-format is not exactly "matching".
    Didn't do any close comparison yet between the standard and expanded DNGs.

    Kind regards.
    Bart ...

  18. #18
    Senior Member dude163's Avatar
    Join Date
    Feb 2011
    Location
    New Brunswick , Canada
    Posts
    1,111
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Thanks, Ill give it a try

  19. #19
    Member
    Join Date
    Nov 2009
    Posts
    74
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    C1 7.1.5 works fine for me but 7.2 skips converted DNG files!

  20. #20
    Member
    Join Date
    Nov 2009
    Posts
    74
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Check out new version of Raw converter here. M8raw2dng – Tool to convert Leica M8 RAW files to DNG

  21. #21
    New Member bla's Avatar
    Join Date
    Feb 2014
    Posts
    4
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Hello everyone,

    I am the developer of m8raw2dng and I want to thank Phil and Scott for their nice write-speed tests. I haven't thought of doing those yet, but it confirms my experience after taking nearly 2k exposures in RAW: You won't miss a shot because shooting RAW is slower (cause it isn't), but it takes longer until you can have a look at the image on the LCD.

    I can tell you that it would be lightning fast if it wasn't for the JPG file in RAW+JPG mode...

    Anyhow, I am always happy to see people testing my tool and feedback is highly appreciated.

    Feel free to contact me and pay a visit to my website for more information. You might want to subscribe to my mailing list to get notified on important updates and new releases.

    Have a nice time and enjoy shooting your M8, especially in RAW mode!

    Arvid
    M8raw2dng - Tool to convert Leica M8 RAW files to DNG

    m8raw2dng.de
    Likes 2 Member(s) liked this post

  22. #22
    Senior Member dude163's Avatar
    Join Date
    Feb 2011
    Location
    New Brunswick , Canada
    Posts
    1,111
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Hi Arvid I tried opening the DNGs in lightroom 3.4 photoshop CS2 and CS 5. but none recognised the file, do I just need newer software? if so Ill try my laptop when I get home it has LR 4

  23. #23
    New Member bla's Avatar
    Join Date
    Feb 2014
    Posts
    4
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Hi dude,

    that hasn't happened before. Can you tell me which parameters you used for conversion? Let's see if we can work that out...

    Arvid
    M8raw2dng - Tool to convert Leica M8 RAW files to DNG

    m8raw2dng.de

  24. #24
    Senior Member dude163's Avatar
    Join Date
    Feb 2011
    Location
    New Brunswick , Canada
    Posts
    1,111
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    I also just redownloaded the latest Adobe DNG codec for win 7

    Adobe DNG Codec 2.0

    and I can view teh DNG files now in windows photo viewer, but they have a slight magenta cast ( Ibelieve there is a command switch to correct that) But they wont open in LR or PS

  25. #25
    Senior Member dude163's Avatar
    Join Date
    Feb 2011
    Location
    New Brunswick , Canada
    Posts
    1,111
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Upgraded to Lightroom 3.6 and it works!

    ISO 2500 summicron 50 1964



  26. #26
    Senior Member dude163's Avatar
    Join Date
    Feb 2011
    Location
    New Brunswick , Canada
    Posts
    1,111
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    NO noise reduction BTW

  27. #27
    New Member bla's Avatar
    Join Date
    Feb 2014
    Posts
    4
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Nice example, thank you!

    Glad you figured it out. I think I know why the older version didn't work. I set the DNG version on conversion to 1.4, and that's most likely not supported by LR 3.4... Do you still have 3.4, so you could test if that's the case? If not, no problem.

    I hope you like what I've done so far and if you got problems or questions, just contact me.

    Cheers,
    Arvid
    M8raw2dng - Tool to convert Leica M8 RAW files to DNG

    m8raw2dng.de

  28. #28
    Senior Member dude163's Avatar
    Join Date
    Feb 2011
    Location
    New Brunswick , Canada
    Posts
    1,111
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    I do have 3.4 still, what did you want me to do

  29. #29
    New Member bla's Avatar
    Join Date
    Feb 2014
    Posts
    4
    Post Thanks / Like

    Re: Unlock 16-bit RAW files for M8

    Well I could give you a modified version per email (for your eyes only )

    And you could convert an image and try then if it works again in LR 3.4.

    But only if you are interested. You are so far the only one having the problem, so I guess most people have moved on to newer versions. But it would be interesting for me of course as I want the tool to be usable for a maximum number of M8 users. That's why I put up with all the Mac/Linux/Windows cr**.
    Arvid
    M8raw2dng - Tool to convert Leica M8 RAW files to DNG

    m8raw2dng.de

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •