The GetDPI Photography Forum

Great to see you here. Join our insightful photographic forum today and start tapping into a huge wealth of photographic knowledge. Completing our simple registration process will allow you to gain access to exclusive content, add your own topics and posts, share your work and connect with other members through your own private inbox! And don’t forget to say hi!

Unlock 16-bit RAW files for M8

M8 is my poor man's MM and I am very glad to stumble into this discovery via this thread http://www.reddotforum.com/showthre...mpressed-format-unlocked-for-the-M8-m8raw2dng

Basically, the sensor data has 16 bit but Leica firmware only saves in 8 bit, throwing away info in the shadow.

Arvid discovered how to set M8 to output 16-bit in Leica's RAW format and wrote a tool to convert the output to DNG. It works for C1, LR, PS Raw Editor. If you use C1, don't use option 'c' when converting RAW to DNG. Very cool.

M8raw2dng – Tool to convert Leica M8 RAW files to DNG


--Cuong
 

scott kirkpatrick

Well-known member
I am very glad to stumble into this discovery via this thread "RAW" uncompressed format unlocked for the M8: m8raw2dng

Basically, the sensor data has 16 bit but Leica firmware only saves in 8 bit, throwing away info in the shadow.

Arvid discovered how to set M8 to output 16-bit in Leica's RAW format and wrote a tool to convert the output to DNG. It works for C1, LR, PS Raw Editor. If you use C1, don't use option 'c' when converting RAW to DNG. Very cool.

M8raw2dng – Tool to convert Leica M8 RAW files to DNG


--Cuong
This is deep in geek territory, and so far only applies to the M8, but I'm intrigued... What these folks have discovered is the "Easter egg" procedure to launch a set of service routines on an M8. And one side effect of the M8 service routine is that it can set the camera to skip the compression step of taking the square root of the intensity signal for each pixel and output the full 12-14 bits of usable data (in a 16-bit field, I presume) as an undocumented RAW file. The site these links take you to also offers software to turn the RAW file back into a useable DNG, without the loss of any precision. I guess they make the files look as if they came from an M9.

Here's how to do it:

If you want to set your M8 to record the RAW files you have to follow the following Steps.

To activate the service mode press

4 times right arrow button
3 times left arrow button
1 time right arrow button
INFO button
Ignore the stuff that appears on screen
Touch the shutter lightly to exit the service menu

Now you can press SET and the compression settings will show an additional option JPG fine+RAW at the bottom of the list. Select it and you’re done.

caveat: the setting will be deactivated on power-off (manual or automatic energy save mode) On next reboot the compression will be set to JPG fine only. Beware of forgetting to change it otherwise you’ll be shooting JPG only… :) .....
Some things to add -- I wouldn't ignore the stuff that appears on the screen -- it is a series of simple and useful tests, some of which might be done in a store to see if service is needed. They show every external control input as received in the camera, for example, the three degrees of shutter press. If a lens is attached the lens code shows, resolving the 6-bit plus 2 bits of frame preview to an 8-bit number (between 0 and 256), as some of us had long suspected. That part seems useful for helping Sharpie-coders get their lines right. You exit each test by pressing SET, but you can try a different one by pressing INFO again. Power OFF and then ON gets you back to normal operation.

I have one M9 which sometimes doesn't do a shutter half-press correctly, and this sort of test would help me nail down the issue. But the M9 "Easter egg" is different, and not known yet. Google hasn't picked it up. Does anyone know how to do this for the later models, and care to share the information???

scott
 

scott kirkpatrick

Well-known member
I haven't tried the 16bit M8 files and their conversion into dng yet. My understanding from what I read is that you shoot after using this trick to set it up, and get L10xxxx.RAW fiiles, and the downloaded program converts these into M9-like .dng files, which any software should be able to render into .jpg or .tif.

scott
 

MaxKißler

New member
...
I have one M9 which sometimes doesn't do a shutter half-press correctly, and this sort of test would help me nail down the issue. But the M9 "Easter egg" is different, and not known yet. Google hasn't picked it up. Does anyone know how to do this for the later models, and care to share the information???

scott
I'd like to know that too...
 
I haven't tried the 16bit M8 files and their conversion into dng yet. My understanding from what I read is that you shoot after using this trick to set it up, and get L10xxxx.RAW fiiles, and the downloaded program converts these into M9-like .dng files, which any software should be able to render into .jpg or .tif.

scott

Yes that's it basically. It stores a 16bitRAW file and a matching jpeg. Run the author's software and it will create a DNG file that is then editable in your software of choice. I use it all the time now with my M8...

The author is also extremely responsive to queries. I found that it would produce files but they were not previewable in Capture One Pro 7 (my RAW/DNG editor of choice for any camera). Using the -p option when creating the files solved the issue for me. See the author's website for description of the command line options.
 
These two images compare the dynamic range between compressed vs. uncompressed RAW on M8. Using the script downloaded from the site, I can't open it with C1 Pro 7. I asked the author and he recommended me to remove the "-c" option and then it works fine with C1.



 

jaapv

Subscriber Member
It still begs the question why Leica decided to disable the uncompressed option on the M8 originally. At the time it was said the camera became too slow to be useable.
Can the bottleneck at the time have been the SD cards available?
 

fotoism

Member
I just did a very unscientific test to get an idea what the write speed difference is between the DNG and RAW+jpeg.

I set the camera to each of the two settings and snap a few shots timing from the time I press the shutter to the time the red LED sops blinking. I am using a Transcend SDHC 16GB C10 card.

DNG on average gives 3.6 to 3.8 seconds.

RAW+jpeg on average gives 10.4 to 10.7 seconds.

It could well be my response time in pressing the buttons and all those possible errors, but at least it shows quite a bit of difference in writing the files.

Can someone else try the same thing with different cards? I don't have any faster card than this so if someone and a faster card and are getting faster speed it would be nice to know.

Or if someone has a proper setup to do this test properly and give some data that would even be better!
 

Knorp

Well-known member
Hi Phil, perhaps you already did without mentioning it, but at least shoot "DNG+JPG fine" so it compares to the "JPG fine+RAW".

Kind regards.
 

scott kirkpatrick

Well-known member
I used to do these tests. To answer Jaap's question, let's use the best card as recommended by Leica at the time the M8 was current. My M8 has an 8 GB SanDisk Ultra, rated at 15 MB/sec, half full. (The SanDisk Extreme cards, although rated faster, ran slower on the M8.) It is set for normal use, discreet advance and auto ISO enabled. Firmware version is 2.014 and it is an M8.U with the shutter modified and the lovely sapphire LED cover. On screen I have online stopwatch running, with a digital time output refreshed every 40 msec.

I set the shutter to "C"ontinuous, and hold it down while focused on the onscreen time indications. For DNG only output, the M8 takes 10 shots at intervals of 0.6 sec, with the times very consistent for the first 5 shots and becoming a bit more erratic, perhaps as long as 0.7 sec on the 9 or 10th shot. Then the internal buffer fills and the time between shots slows as the camera must wait for internal buffer space to be available. The first buffer-full shot is delayed by an amount that varies from one series to the next, but the second and further buffer-full shots are separated by a consistent interval. Finally, I release the shutter and watch the red light blink, taking one more shot when it stops to record that time. The initial interval between shots (when there is plenty of buffer space) is the same whether I am shooting DNG only, DNG + fine JPEG, or RAW plus fine JPEG -- 0.6 sec. So this must be time needed to get 20 MB of unprocessed data off of the CCD. Other processing will proceed in parallel, but the camera is ready to take another shot.

The next stage, shooting with the buffer full and needing to be drained a bit, does depend on how much data in final form will be written to the memory card. For DNG only, the intervals become 2.7 to 2.8 sec per frame. For DNG + fine JPEG, they range from 7 to 7.5 sec per frame. But for RAW plus fine jpeg they are only 5.5 sec, or about twice the time required to clear buffer space for a compressed DNG file.

The final interval also depended on the type of data being written. To clear the entire buffer and stop the red light blinking took 22 sec with 8bits per pixel DNG files, 55 sec with the RAW data + fine JPEG, and 75 sec with compressed DNG + fine JPEG.

To really understand why the RAW format wasn't used we still have to understand what the actual parallel processes are that are performed after the data is in the buffer and whether the camera only writes to the chip when done, performing all processing in the internal buffer, or uses the chip as an intermediate store. I'm still thinking about that.

scott

PS: Checking the files that were written, I see that the case I thought was RAW + fine JPEG was just jpegs. I'll go back and see what I did wrong so that the RAW selection didn't take hold. So now the relative times are DNG only 22 sec to process the last shot and clear the entire buffer, fine JPEG processes the final shot and clears the entire buffer in 55 sec, and doing both (compressed DNG plus a fine JPEG) takes 75 sec to process the last shot and clear the entire buffer. Note that the time for fine JPEG + DNG is about equal to the sum of the times for each one separately. There is only one signal processing chip to do both jobs, so this is understandable.
 
Last edited:

fotoism

Member
Hi Phil, perhaps you already did without mentioning it, but at least shoot "DNG+JPG fine" so it compares to the "JPG fine+RAW".

Kind regards.
Knorp:

Actually I have not tried that because I only shot RAW, and I was only interested in how this RAW file manuever would impact my shooting.

However, you are right, a comparison between RAW+jpg fine and DNG+jpg fine would indicate the difference in RAW vs DNG write time better, because both are also writing JPG fine at the same time.

So I tried again this morning and I got 8.3 to 8.5 seconds for DNG+JPG fine, which is much closer to the RAW+JPG fine time.

I am wondering if the UHS class 3 would work on the M8 at all. It supposedly has a minimum bus speed of 30MB/s as compared to the C10's 10 MB/s.
 

scott kirkpatrick

Well-known member
I see that fotism's 10 MB/sec Transcend card clears a DNG + fine JPEG in 8+ sec while my 15 MB/sec Ultra takes 7.5 sec. His DNG only takes 3.5+ sec; with my card it took 2.8 sec. Those results seem consistent with the quoted speeds of the two cards. The Extreme cards are now said to run at 30 MB/sec. I have used that line in M8s, so they work, but they were slower.

Notice that the buffer fills at 10 shots, regardless of whether you are shooting DNG only, DNG plus JPEG, or JPEG only. That suggests that most of the interesting things are happening as the data is written to the card, not reserving extra memory in the internal buffer for the JPEG.

scott
 

fotoism

Member
I ran some tests using the C mode and these are my numbers:

10 shot RAW+JPG fine 1:27 to 1:29 for the red LED to stop blinking,:eek:
10 shot DNG+JPG fine 1:08 to 1:11,
10 shot DNG only 25 to 26s

When shooting in RAW+JPEG fine the shots were fired with the best rhythm and at a faster pace. The blinking was long, though. The other two showed slower pace and the shot to shot intervals are a bit irregular.

I also found that if I keep on pressing the shutter button in DNG doing C mode, it was able to squeeze in an extra shot or two, after a very brief hesitation, before the "buffer full" red box shows up. That may be due to my card being slower and there is time for the buffer to clear and make room for another shot. I was able to get a maximum of 13 shots in one go in DNG only but only 10 on the other two formats.
 

scott kirkpatrick

Well-known member
I rechecked my results for timing fine JPEG alone, and then got a series with RAW + fine JPEG. Again the only differences are seen once the buffer is full.

For the first 10 shots, with shot intervals only due to writing into the buffer:
DNG only .62 sec between shots
JPEG only .62 sec
DNG + JPEG .67 sec
RAW + JPEG .55 sec (!!)
(So if you don't fill the buffer, RAW is great!)

Once the buffer was full, shot intervals become:
DNG only 2.7 to 2.8 sec
JPEG only 5.5 to 5.6 sec
DNG + JPEG 7 to 7.5 sec
RAW + JPEG 9.6 to 10.2 sec.
(So the time to output a RAW is about twice the time to output a compressed DNG.)

and time to clear a full buffer by writing out 10 exposures:
DNG only 22 sec
JPEG only 53 sec
DNG + JPEG 74 sec
RAW + JPEG 98 sec
(again each RAW takes twice as long to output as DNG)

scott
 

Knorp

Well-known member
So far my files are roughly 21Mb. Still having a bit of difficulty with C1P7.2 as not all converted files show up.
Apparently, whereas Aperture seems less picky btw, the files' DNG-format is not exactly "matching".
Didn't do any close comparison yet between the standard and expanded DNGs.

Kind regards.
 
Top