How do you measure the Dynamic Range of a Camera?

[pophoto]

Hi Guys,

I'm very curious to know how any of you can measure the Dynamic range of a Camera or digital back?

I know most MF maketing do tend to say they have a range of a bout 12-stops, but how is this measured and can it be repeated in real use? Although I do believe I see this in MF photos or at least increased DR over 35mm, I just want to know this isn't multiple shots combined, which I doubt!

Thanks,
Po

 

[dougpeterson]

It's very hard to come to a consensus on a particular numerical value.

Does it count as "dynamic range" if you can tell a darker tone from a lighter tone with 51% accuracy, the color is only vaguely accurate, and the noise looks like a series of spilt water colors on a black tshirt? Does it count as dynamic range if you can see a bit further into the cloud but there are several bands of false color in the transitions between the bright part of the cloud and the super bright part of the cloud?

Some people would say yes, because that's more-or-less (greatly simplifiying it for the sake of a practical/short answer) the engineering definition. Typically using a stepper wedge (a piece of paper with a series of black-to-white steps) under controlled light.

HOWEVER, if you're a photographer and not an engineer it's actually relatively easy to compare dynamic range between cameras. Just take both into a difficult scene like a mid-day landscape or cityscape and take a series of exposures in 1/3 stop increments. Bring them all back to the computer and load them into your prefered software (which does play a pretty drastic role, so if you're really interested in getting good DR you should try several softwares and in the end pick the one that balances good results with usable workflow for your needs). In the software find the brightest exposure in which important highlights in the scene can bepleasently maintained (accurate color, nice transitions - i.e. you'd be happy printing it) using a combo of highlight recovery, exposure compensation and whatever other tools are available in that software (e.g. in Capture One you could select the "linear" profile curve).

Take those same images and bring the shadows up as far as they'll go and still look pleasent. Note that you may want to accept some noise/grain and that you should judge that based entirely and only on what you would be happy printing/delivering-to-a-client/posting-to-you-website/whatever. You should also play with the noise reduction tools as the default values may not be optimal.

Then compare the shadows in these two images to each other. Which one shows an absolute higher level of detail (literally in which image can you see more real-world objects in deeper shadows)? In which image are the colors of the shadows pleasent and consistent? In which image is the noise structure more pleasent to your eyes? In which image are the transitions from deep shadow to quarter-tones more natural, smooth, and photographic in appearance?

That will tell you which one has better dynamic range as measured by the only criterium that matters to me: the appearence of real world images.

So my suggestion would be avoid many pages of fruitless engineering banter and don't worry about the numbers anyone claims (manufacturers, well-intended photographers, websites like dXo) and if/when it's part of a decision you need to make (to rent/buy/sell) work with someone (note this is a very self-interested suggestion) who will help you with the test I outlined above.

Doug Peterson (e-mail Me)
__________________

Head of Technical Services, Capture Integration
Phase One Partner of the Year
Leaf, Leica, Cambo, Arca Swiss, Canon, Apple, Profoto, Broncolor, Eizo & More

National: 877.217.9870 *| *Cell: 740.707.2183
Newsletter | RSS Feed
Buy Capture One 6 at 10% off

Masters Series Workshop:
New England Landscape - Fall Color (Oct 5-8)

 

[Graham Mitchell]

Technically it's a measure of the ratio of the noise floor to the maximum captured signal, but as Doug pointed out, the issues of shadow noise can make the setting of the lower limit a matter of contention.

 

[Thierry]

Or another way, completing Doug's way, would be, when taking such images from a scene, to use a spot-meter and to measure the light from the deepest shadows to the brightest highlights.
Then take these images and work on the highlights and shadows as suggested by Doug.
By doing so you would also have an idea of the DR in f-stops.

Thierry

 

[cunim]

So my suggestion would be avoid many pages of fruitless engineering banter and don't worry about the numbers anyone claims (manufacturers, well-intended photographers, websites like dXo) and if/when it's part of a decision you need to make (to rent/buy/sell) work with someone (note this is a very self-interested suggestion) who will help you with the test I outlined above.

Great post, Doug. There are endless pages of DR discussion over at LULA, because photographers and technical users have completely different expectations. As you point out, a photographer wants a pleasing image over a broad instrascene luminance range with large areas of light and shadow. Under these conditions, lens flare (primarily) destroys precision so intensity rendering in shadows and highlights becomes poor. A technical user (e.g. fluorescence microscopist or astronomer) is usually imaging a very narrow instrascene luminance range with only very small areas of high brightness. Under those conditions, high precision is more easily maintained into highlights and shadows. I have made images with true 16 bit precision, but only from cooled or amplified cameras and self-luminant targets. In contrast, I would be happy to get decent looking photographs over a 1000:1 luminance range. Aargh. I would be happy to just get decent looking photographs.

 

[Shashin]

You need to put dynamic range in contex. It measures limits. There are many situation where when the scene is exaclty the dynamic range of the camera, that even with a perfect exposure, you simply will not like the results and imagine the data to be clipped--most scenes are not going to look pleasing if values hit the black point and white point.

DxO score are useful as a measure of performance among different cameras/sensors. This helps you compare them. They are right (and how DR is obtained is clearly documented on the site). The scores are also "real world" in that it shows the actual response of the sensor.

The folks who complain that the test and methodology that make all these wonderful devices we use cannot be used to evaluate the same devices basically don't understand the tests. So if you are getting what you think is a different result, then perhaps the "mistake" does not rest with the test. I really love how folks "proclaim" the "actual" DR of their system, but yet show no more methodology than simply taking a picture of a scene which has never been measured.

Now, if you are trying to figure out how your camera works and whether it will give you results under certain conditions, then you certainly can test for this by simply taking pictures--you will get no quantitative data with which to measure DR, not will it show DR. If as Theirry suggests by measure scene luminance, you can start to quantify the results, but still it is not a measure of DR, just quantification of scene luminance range that result in pleasing images for you with the system you are using.

Photography is a process. I find it important for the photographer to go out and try to start understand or see how their system is going to work with the environments they work in. DxO data is really only a data point used for comparing cameras you are interesting in purchasing.

 

[Guy Mancuso]

My issue with DXO is it does not take into account of the raw processing which to me the FINAL result after processing is the meat of the matter. We all know that there are special recipe's built into the file algorithms in each raw processor and we never see the true potential of a raw file until it hits the raw processing software. To me DXO is just a guide not a religion. IMHO it is what is delivered or goes to print is the most important factor but this comes from a artist not a scientist. I leave that stuff up to them to argue. I know when to get off the bus. LOL

 

[yaya]

To illustrate Doug's explanation, here's an image that represents a high dynamic range scene of about 11 stops

it is about 2/3 of a stop over exposed to allow for cleaner shadows

The histogram on the Left shows the captured data in f-stops on the horizontal axis. "0" will normally be where a well exposed 18% Grey card will be seen and ideally you will want the Right edge to be around "+2"

In the first image I used a standard contrast curve and in the second one I used a more linear one, lifting the shadows and bringing the highlights in.

The vertical axis shows the 8-bit output values (most of us are familiar with 0-255 in Photoshop)

Over doing this will result in an unpleasing effect, which you sometimes see in HDR images.

Capturing a high dynamic range scene with a device that can record all this data allows you to "clip" or "stretch" the bright/ dark areas to be pleasant and to be properly translated to your choice of output media

For the 3rd image I've created a "S" curve which to my eye seem to show a good balance between the sky, the shadows and all the tones in between

 

[pophoto]

Thank you for all the explanations guys, and Doug for your detailed outline. Very much appreciated.

For me, it is also more important to SEE it artistically as well. I just want to come to terms on how other photographers see or interpret DR and technically how you can see this as being true in your own systems as well for what might matter.

Then I am also curious now, do you guys see dramatic improvements in DR with MF over 35mm? (I know this is also a typical question) I know some of you have talked about 1/2 stop improvements with the IQ180 over the P65+, so this also got me thinking and wanted to better understand this or similar statements.

 

[dougpeterson]

The folks who complain that the test and methodology that make all these wonderful devices we use cannot be used to evaluate the same devices basically don't understand the tests. So if you are getting what you think is a different result, then perhaps the "mistake" does not rest with the test. I really love how folks "proclaim" the "actual" DR of their system, but yet show no more methodology than simply taking a picture of a scene which has never been measured.

I never said they can't be used to evaluate them. dXo does an extremely fine job of outlining an quantitive-only based approach to evaluate the dynamic range of a digital sensor through generic raw-processing and based on the quality of the other products/services they provide I think it's safe to assume they do a fine job executing them.

However a quantitive only analysis which ignores the importance of proprietary software, proprietary data in the raw file (e.g. dark frames), the impact of lenses and internal body baffling, and the dozen other variables that an actual picture introduces will only tell you a very small part of the story.

Qualititive analysis will not produce a numerical value (though as smartly suggested above you can backwards-engineer it using a spot meter), but it does provide in my experience the best method to compare two systems.

Notably I feel confident saying the dXo staff would be the first to tell you that their test will not indicate the aethetics of the noise or the impact of other variables as outlined above.

But thanks for the implication that I don't "understand" dXo's methods .

Quantative-only analysis will tell you that Sound A and Sound B are the same volume (absolute decibles) but a qualitative analysis will tell you that one is a jazz section solo and another is a the sound of ten babies crying. If the question is "how pleasent is Sound A vs. Sound B?" then the quantitive-only analysis will only go so far .

Most photographers are asking "how much range can I pleasently render from a captured image using camera X compared to camera Y" are IMO much better served with a qualititative analysis.

 

[dick]

Hi Guys,

I'm very curious to know how any of you can measure the Dynamic range of a Camera or digital back?

I know most MF maketing do tend to say they have a range of a bout 12-stops, but how is this measured and can it be repeated in real use? Although I do believe I see this in MF photos or at least increased DR over 35mm, I just want to know this isn't multiple shots combined, which I doubt!

Thanks,
Po

... It's like the published fuel consumption figures for a car - they might never be achieved in normal use, and it depends how you drive your car, but they are a standardized yardstick.

If you work in a studio you light he scene to give the result you want with your kit, but if you do weddings or black cats in snowdrifts, DR is important.

...we normally have plenty of cloud for fill in the UK, but landscapes could be a DR problem in Spain or Arizona?

¿Why bother measuring DR?

...to assess how a demo camera would perform for you for your use.

...to assess when you would benefit from using (more) fill, e.g. fill-flash or reflectors.

If you no not have a hand-held spot meter with you, how do you asses the DR of a scene?

...put the camera in manual mode, with TTL spot metering, and use your camera as a hand-held spot meter?

 

[yaya]

If you no not have a hand-held spot meter with you, how do you asses the DR of a scene?

...put the camera in manual mode, with TTL spot metering, and use your camera as a hand-held spot meter?

you can use the spot meter on the digital back's display, it'll give you an exact reading in f-stops and show you where the spot sits on the histogram

 

[Lars]

:thumbs:Very interesting and informative reading, thanks everyone for contributing!

 

[dick]

you can use the spot meter on the digital back's display, it'll give you an exact reading in f-stops and show you where the spot sits on the histogram

Do all digital cameras do this? ...or just all MFDs?

Most have the facility to warn you of under or over exposed areas - and tell you where they are.

 

[Lars]

Do all digital cameras do this? ...or just all MFDs?

Most have the facility to warn you of under or over exposed areas - and tell you where they are.

I've never used a camera that indicates underexposure areas. That would be useful.

Nikon shows luminance overexposure, not as useful as channel overexposure.

 

[yaya]

Do all digital cameras do this? ...or just all MFDs?

All MFDBs that have a 3.5" touch screen that's operated with a stylus and that can show a histogram that is generated off of the raw data;-)

 

[GrahamWelland]

All MFDBs that have a 3.5" touch screen that's operated with a stylus and that can show a histogram that is generated off of the raw data;-)

I wonder which ones they might be?

 

[wentbackward]

Do something like this with a grey card. You determine where your sensor will clip and you can boost the darker ones to see where noise becomes unacceptable. In my experience the DR claims are generally rubbish, but depends a lot on the scene and your intentions. What it means is Ansel Adams is still (perhaps more) right about pre-visualisation in a digital world.

 

[WWLEE]

I'll probably get a spanking for asking this, but what do you make of the following DXO review of the IQ180? Either I'm not understanding something or I was just uninformed, but I thought the IQ180 had the highest DR of any camera, yet below they say it is less than some APS-C cameras.

Friday October 14 2011
When we received the IQ 180 Digital Back from Phase One, we expected a lot from this huge 80 MPixels sensor. And, indeed, the sensor is the best we ever tested with an overall score of 91: the best score ever and the first one over 90 points on DxOMark scale! Let’s check the details.

As usual with Phase One sensors, we were particularly impressed by its Color Depth with a portrait score of 26.5, also the best Color Depth ever measured on DxOMark.

In term of dynamic range, the IQ180 also reaches the very high score 13.6 Evs, but still doesn’t beat the best APS-C sensors, like the Pentax K5 (14.1 Evs) or the Nikon D7000 (13.9 Evs).

Thanks to its specific Sensor Plus mode, the IQ 180 scores pretty good in low-light condition with a 966 ISO score, but it remains far from the best cameras like the Nikon D3s with a 3253 ISO low-light score.

So, as the IQ180 will mainly be used in a studio with full control of the lighting conditions, the low light ISO weakness shouldn’t be an issue. And, the awesome color depth and dynamic range should give full satisfaction to the most demanding photographers.

The IQ 180 vs other Phase One cameras:
In this comparison with its predecessors the Phase One P65+ and Phase One P45+, we can see that the IQ 180 model improves every aspect except Low-Light ISO Score. This can be explained by the fact that the sensor surface didn’t change.

Looking at the non normalized results, it is still a bit surprising that we did not see any specific improvement of the Dynamic Range.

Phase one IQ180 vs the best Full Frame and APS-C:
In this comparison of the Phase One IQ 180 vs Nikon D3x and Canon EOS 1Ds Mark III or with the APS-C Nikon D7000 and Pentax K5, we clearly see the superiority of the IQ 180 in term of Color Depth, its good Dynamic Range and its relatively poor Low Light ISO performance. But as we said, these strength and weaknesses are very well balanced for a studio camera.

Overall this is quite an achievement for Phase One managed to increase the DxOMark score of the IQ 180 compared to the P65 Plus while adding 20 MPixels within the same sensor surface.

It is still interesting to note that theoretically, the Phase One IQ180 could score much better with such a huge sensor surface, if the pixel quality would be closer to the quality of the best APS-C (or full frame) sensor’s pixel.

 

[WWLEE]

I'll probably get a spanking for asking this, but what do you make of the following DXO review of the IQ180? Either I'm not understanding something or I was just uninformed, but I thought the IQ180 had the highest DR of any camera, yet below they say it is less than some APS-C cameras.http://forum.getdpi.com/forum/images/smilies/wtf.gif

Friday October 14 2011
When we received the IQ 180 Digital Back from Phase One, we expected a lot from this huge 80 MPixels sensor. And, indeed, the sensor is the best we ever tested with an overall score of 91: the best score ever and the first one over 90 points on DxOMark scale! Let’s check the details.

As usual with Phase One sensors, we were particularly impressed by its Color Depth with a portrait score of 26.5, also the best Color Depth ever measured on DxOMark.

In term of dynamic range, the IQ180 also reaches the very high score 13.6 Evs, but still doesn’t beat the best APS-C sensors, like the Pentax K5 (14.1 Evs) or the Nikon D7000 (13.9 Evs).

Thanks to its specific Sensor Plus mode, the IQ 180 scores pretty good in low-light condition with a 966 ISO score, but it remains far from the best cameras like the Nikon D3s with a 3253 ISO low-light score.

So, as the IQ180 will mainly be used in a studio with full control of the lighting conditions, the low light ISO weakness shouldn’t be an issue. And, the awesome color depth and dynamic range should give full satisfaction to the most demanding photographers.

The IQ 180 vs other Phase One cameras:
In this comparison with its predecessors the Phase One P65+ and Phase One P45+, we can see that the IQ 180 model improves every aspect except Low-Light ISO Score. This can be explained by the fact that the sensor surface didn’t change.

Looking at the non normalized results, it is still a bit surprising that we did not see any specific improvement of the Dynamic Range.

Phase one IQ180 vs the best Full Frame and APS-C:
In this comparison of the Phase One IQ 180 vs Nikon D3x and Canon EOS 1Ds Mark III or with the APS-C Nikon D7000 and Pentax K5, we clearly see the superiority of the IQ 180 in term of Color Depth, its good Dynamic Range and its relatively poor Low Light ISO performance. But as we said, these strength and weaknesses are very well balanced for a studio camera.

Overall this is quite an achievement for Phase One managed to increase the DxOMark score of the IQ 180 compared to the P65 Plus while adding 20 MPixels within the same sensor surface.

It is still interesting to note that theoretically, the Phase One IQ180 could score much better with such a huge sensor surface, if the pixel quality would be closer to the quality of the best APS-C (or full frame) sensor’s pixel.