Maybe flawed is a strong word on my part. How about, "not photographically relevant". They measure DR where the signal-to-noise is 0dB - noise and scene detail are equal magnitude. This is a totally unusable shadow for a photographer - you'd never even try to use such a thing. 0dB is a nice arbitrary engineering measure, and one I understand is even used in many sensor datasheets. So it isn't like they are doing anything wrong using it.
But I don't see this as an issue because the measurements are all equal for all cameras. So if a camera is ranked at 12 stops, and another at 10, I certainly would expect a comparative difference here.
The rub is that if the SNR curves were linear it wouldn't matter where they chose. 0dB, 10dB, -10dB - whatever - we are only comparing cameras and all that matters is the comparison, not the absolute value. The problem is that the curves are not linear at all. And at least at the moment with the technology right at base ISO many cameras have a bend in their SNR curve near this 0dB point. So what happens by choosing a "really noisy shadow" as their reference point they give more benefit to a camera with really low read noise than is really useful to a photographer.
But again, a camera with a sensor with a really low read noise can certainly have a lot of DR. If you expose to the right you are using it. Or by under exposing and developing up. This would move the DR to the highlights and cut in the shadows but if the shadows are that good you can afford to without much impact to image quality (specific case: K-5 sensor for example)
If they chose a higher SNR as their reference point (6dB, 10 dB?) the GX1/G3 and GH2 sensors would have closer DR. You can see this if you look at their "Full SNR" curve tab (not available in the compare feature, must go to the individual camera reviews). Imatest provides "DR" measurements for a few different SNRs with some different number crunching to try to be a bit more "photographically relevant".
But that would mean then that they are ignoring the DR in the shadows. Again, low shadow noise is equal to DR you can use- either when you expose to the right or under expose and develop back up. And when you do the last you move the whole DR scale to capture more highlight range. This is what the Pentax/Canon call "highlight priority" (er.. highlight correction in Pentax). Sony allows the same.
With Olympus coming the E-30, they are doing this but other than setting the camera to ISO 100 (when available) you are stuck with very noisy shadows as the whole DR scale was prioritized for highlights at all times.
So I still don't understand how DXo doing it that way still fails to account for real world photographic DR we can use.
That's right. DxO just measures between a saturated sensor and an SNR of 0dB, so they get both shadow and highlight in one measurement - the choice of the grey point is arbitrary. The only problem is 0dB is just a bit too low to measure if our goal is to compare "usable" DR, as described above. When they started doing their measurements it was probably a non-issue, but now that we've got some incredibly low read noise sensors out there it is skewing the results in a way not practical to photographers.
Once again, that is indeed part of DR. I can tell you right now the K-5 has bags of shadow DR and I can shift it at will to get highlights, making it quite real world usable indeed.
DPReview has their studio shots, and they include a shadow box that is good for pushing to see how the shadow noise looks. The RAW files are downloadable. The problem is they really don't have a good highlight feature anywhere in the scene to make sure the camera hasn't just traded shadow range for highlight range. You can kind of get around this by verifying that the exposure was the same and then go look at DxO's ISO measurements for the cameras. Their ISO measurement is in fact just a measurement of the saturation level of the RAW file. By noting the exposures used in the DPR test and checking the DxO ISO measurement you can adjust how much you push the shadows for each camera for a "fair" comparison.
Well in general I have measured DR relative to other cameras my own way with several Olympus cameras (e-300, e-330, e-410, e-420, E-3, e-620) and other brands (LX3, LX5, K-5, Pentax Q) and the numbers that DXo has have pretty much always corresponded to a certain level of "where do these cameras stand" relative to each other's DR.
They don't, but they do have nearly the same saturation level (adjusted for sensor resolution differences) - by which I mean the maximum number of photons that can be collected before clipping. The measured ISO of the sensor relates to efficiency of course. However, if what we really want is nice clean mid-tones and the freedom to push and pull an exposure without noise showing up everywhere (not just the shadows) then we want to gather as many photons as possible so we will have the lowest photon shot noise. And this is set by the saturation level, which is just about the same for all the cameras (the GH1 is the odd man out here, slightly higher than the others but only by 1/3 of a stop).
Efficiency measures how fast "the well" fills up. Saturation measures how deep "the well" is. For best low ISO noise performance we want the deepest "well" possible but don't particularly care how fast it fills. For best high ISO performance our concerns are exactly the opposite - it could be a shallow well (we won't ever get to collect many photons anyway at high ISO) but we want it to fill efficiently.
Ken
Hmm ok.
- Raist