New IQ4 Feature - Frame Averaging

Don't take "technical" explanations too seriously. After the trichromatic howler of disjoint transmission curves for the color array (under the technical explanation section), I don't think we have to debate before vs. after A/D conversion. The idea of an analog computer at each pixel doing the averaging? Sounds wrong to me, but I'm not a chip designer. I thought A/D was done on-chip and only a digital signal read off of it. In which case, averaging on the CPU makes a lot more sense, as all the circuitry to do that is already in place, and it's just software.

Here's a question: To get smooth water, you can take a long exposure, or average many short exposures. How much effect do gaps between the shorter exposures have? Star trails are badly affected by gaps, but averaging waves need not be. Motion being (mostly) continuous, a long exposure will do better, but by how much? Suppose I take 100 1/10 second exposures one second apart? How does that compare to a 10 second exposure? 100 seconds?

I should just do the experiment....

I do frame averaging quite a bit for astro images but I've also done a few landscapes that way to simulate a long exposure. I like keeping the exposures separate because I like to experiment with different blending methods.

It does seem to me that averaging and doing a true long exposure will give different results. A long exposure accumulates light uniformly over time and once the spot in the frame (pixel) has been lightened it won't be un-lightened later. It seems intuitively that it would be more like blending exposures in Lighten blending mode rather than averaging.

The example here uses both averaging and Lighten blend mode on a set of 48 frames (not necessarily shot immediately consecutive). These were all shot with a Sony a7r2.

First, here is one of the frames by itself.


This one shows combining frames in Mean stack mode


This one shows the combination in Lighten blending mode


I ended up liking the sky better in Mean stack mode but I like the effect on the water using Lighten blending mode so I ended up blending the two together in Photoshop.

Hi Craig,

Great shot, I also have used the various Stack modes in CC for night work, but never thought about doing it in daytime work. Great shot and it have given a new thing to try out.

Paul C

Quoting from the article:

At its heart, this tool works by averaging two or more (often many more) sequential captures together, before the raw file is generated. This has the effect of evening out noise in the shadows. With two shots the noise should be roughly half as much as a single capture (which is already extraordinarily low), with four captures it should be roughly half as much again, and yet again half at eight captures.

Click to expand...

If that rate of improvement is correct, why are Phase One first implementing it in a camera? I am sure the patents would be worth an enormous amount more to anyone producing NMR or MRI machines, as well as plenty of other scientific and medical systems! :ROTFL:

Averaging frames is, at it's heart, a basic signal averaging exercise, and the signal-noise ratio improves with the square-root of the number of signals, so it should be 4 captures to half the noise, 16 captures to quarter it, etc. This also only applies to "random" noise, if the sensor suffers from systematic noise, it wouldn't be reduced by frame averaging.

That said, I assume in this case Phase One have either failed to explain the feature, or have failed to actually proof-read the marketing sheet for technical accuracy.

RobbieAB said:

Quoting from the article:


If that rate of improvement is correct, why are Phase One first implementing it in a camera? I am sure the patents would be worth an enormous amount more to anyone producing NMR or MRI machines, as well as plenty of other scientific and medical systems! :ROTFL:

Averaging frames is, at it's heart, a basic signal averaging exercise, and the signal-noise ratio improves with the square-root of the number of signals, so it should be 4 captures to half the noise, 16 captures to quarter it, etc. This also only applies to "random" noise, if the sensor suffers from systematic noise, it wouldn't be reduced by frame averaging.

That said, I assume in this case Phase One have either failed to explain the feature, or have failed to actually proof-read the marketing sheet for technical accuracy.

Click to expand...

I'm pretty sure it's safe to assume that a company that develops these kind of high end electronics employs people that have a basic understanding of SNR.

Craig Stocks said:

It does seem to me that averaging and doing a true long exposure will give different results. A long exposure accumulates light uniformly over time and once the spot in the frame (pixel) has been lightened it won't be un-lightened later. It seems intuitively that it would be more like blending exposures in Lighten blending mode rather than averaging.

Click to expand...

If there really is no gap between the exposures, the result should be the same as with a long exposure. If a bright highlight would be in only one of the separate frames, it would be a highlight in the long exposure either. If you expose correctly, it would take the entire long exposure to get a highlight.

JeRuFo said:

I'm pretty sure it's safe to assume that a company that develops these kind of high end electronics employs people that have a basic understanding of SNR.

Click to expand...

So why are the publishing a claim that, if true, would in other industries, I suspect, be worth many times the value of the entire 645 digital MF market?

NMR, which is a technique I have used, relies heavily on SNR. Being able to improve SNR linearly with sample count would revolutionise it, as it would open up a whole range of experiments currently impractical due to the exponential sample count needed for averaging. As this would also govern MRI (NMR without a nuclear in the name), that would be huge. And that is the one technique I remember well from my days in the Lab, I am sure there are others! Either it is not a simple frame averaging, or something doesn't seem right.

It's also worth pointing out, which the marketing script naturally wouldn't, that frame averaging methods only work against random noise. It's a minor nit-pick, on it's own not worthy of comment, but even with effectively zero random noise, if there are non-random noise sources, they will still be present, just "clearer".

So, presented with a scenario that work included in the new feature is going to win either a Nobel Prize, or a Fields Medal (or both!), and the possibility someone screwed up the marketing script, I know where my money is.

Note: Messed up marketing script doesn't mean the engineers are incompetent, it means someone in the marketing department didn't understand what they were told by the engineers.

Relax.

It's Marketing.

They've been misrepresenting the technical details of every product since the advent of oxygen breathing organisms. There are no exceptions. Think of it from their perspective: If they DON'T change what the engineers write, they aren't doing anything to justify their jobs.

--Matt

A background in low signal image analysis makes one dubious of methods that claim noise reduction rises linearly with frame number. I had a quick look at the zero noise method, but I have very limited understanding of Bayer processing so exactly what is going on there is fuzzy. I do know there ain't no free lunch.

It seems that, contrary to what is implied in the marketing material, averaging is not "on the chip" but "in the raw". In other words, multiple readout does occur but Bayer processing does not - until a final raw file is generated. Readout noise would still be reduced exponentially, not linearly.

It also seems that the method relies on substituting noise properties from areas of high signal (overexposed image) into areas of low signal. If that is true, the method would not work for cases where high levels of noise are present at long exposures. Not suitable for really dim conditions, but most photographers don't work under really dim conditions so that may be OK.

I have no idea if my understanding is correct and I would like to understand this better. Perhaps Phase could provide a clear explanation of this potentially important feature, and compare it to traditional frame averaging.

Since the technique is still in the development phase, I doubt they'll want to have a heart-to-heart about it with people outside the company. You'll be allowed to test its effectiveness, but it probably stops there or very shortly beyond that point.

As pointed out here the article contained an error: the decrease in noise is expected to be at the typical frame averaging rate of roughly 1 stop of noise reduction with four reads and 2 stops of noise reduction with sixteen reads. Sorry for the error; I've corrected it in the article.

I've added a summary statement in the article that reads:
"Frame averaging techniques are possible with any camera. But frame averaging often call for many frames, even dozens or more for some use cases. So having the IQ4's ES do the heavy lifting in an automated and user-friendly way, in camera, is transformative to the use of this technique. Doing frame-averaging manually is possible, but highly tedious and time consuming. All indications are the IQ4 tool will make it just another technique that can be easily applied to specific kinds of scenes. In this way it's reminiscent of the automated Focus Stacking Tool in the XF; focus stacking is possible with nearly any camera, but after you've done a full day of focus stacking with the XF it's unimaginable to do it on another platform."

MGrayson said:

Relax.

It's Marketing.

They've been misrepresenting the technical details of every product since the advent of oxygen breathing organisms. There are no exceptions. Think of it from their perspective: If they DON'T change what the engineers write, they aren't doing anything to justify their jobs.

--Matt

Click to expand...

Good point.

One might consider that this article was written by Doug, not by Phase Ones marketing dept. He’s proven very adept at the technical aspects before, based on his last post it sounds like this was an oversight and miscalculation.

Wayne Fox said:

Good point.

One might consider that this article was written by Doug, not by Phase Ones marketing dept. He’s proven very adept at the technical aspects before, based on his last post it sounds like this was an oversight and miscalculation.

Click to expand...

Only if one had read the article. :facesmack: I saw only a description of the error. Of course, anyone can make a mistake in print (I’ve made hundreds), and Doug is definitely on the side of truth and accuracy. My statement about marketing stands, but I apologize to Doug for tarring him with that brush.

Matt

MGrayson said:

I apologize to Doug for tarring him with that brush.

Click to expand...

Apology easily accepted. But next time bring feathers; otherwise, what's the point?

dougpeterson said:

As pointed out here the article contained an error: the decrease in noise is expected to be at the typical frame averaging rate of roughly 1 stop of noise reduction with four reads and 2 stops of noise reduction with sixteen reads. Sorry for the error; I've corrected it in the article.

Click to expand...

Thanks, Doug. That is very clear now. A convenient way of doing standard frame averaging in camera. That is certainly worthwhile.

Of all the IQ backs since my 180, this is the first that has offered enough new capabilities (better color cast uniformity, higher res, frame averaging, ES) that I would consider an upgrade. Each of those capabilities does something to make workflow easier and/or the image better. Any two of those would not be enough, but all four .... oh yes and automated stacking. Hmmmm.

dougpeterson said:

As pointed out here the article contained an error: the decrease in noise is expected to be at the typical frame averaging rate of roughly 1 stop of noise reduction with four reads and 2 stops of noise reduction with sixteen reads. Sorry for the error; I've corrected it in the article.

I've added a summary statement in the article that reads:
"Frame averaging techniques are possible with any camera. But frame averaging often call for many frames, even dozens or more for some use cases. So having the IQ4's ES do the heavy lifting in an automated and user-friendly way, in camera, is transformative to the use of this technique. Doing frame-averaging manually is possible, but highly tedious and time consuming. All indications are the IQ4 tool will make it just another technique that can be easily applied to specific kinds of scenes. In this way it's reminiscent of the automated Focus Stacking Tool in the XF; focus stacking is possible with nearly any camera, but after you've done a full day of focus stacking with the XF it's unimaginable to do it on another platform."

Click to expand...

Hi Doug,

Apologies, I wasn't trying to be rude, and I may have gotten overly defensive with my second post.

That makes much more sense to my reptilian long ago scientist brain.

R.

Maybe I'm wrong but from my perspective it just seems like Phase One is resorting to gimmicks now.

DB5 said:

Maybe I'm wrong but from my perspective it just seems like Phase One is resorting to gimmicks now.

Click to expand...

Other than as 'something new' to differentiate the product (beyond simply MP), it does seem rather pointless in the DB itself.

Having used Lujik's original Zero Noise, the technique is far better suited to post (i.e. big screen + cup of coffee + music), so if you're listening P1, just stick it in C1 where it belongs.

DB5 said:

Maybe I'm wrong but from my perspective it just seems like Phase One is resorting to gimmicks now.

Click to expand...

That simply means that your perspective is one of a photographer for whom this technique is not relevant for what/how they shoot. Totally okay.

Much like Electronic Shutter, Focus Stacking, Profoto Air integration, Vibration Delay, Exposure Heat Map, Exposure Calculator... tools that are incredibly useful to a narrow range of users.

RobbieAB said:

Apologies, I wasn't trying to be rude, and I may have gotten overly defensive with my second post.

Click to expand...

Totally fine. You were 100% right and your correction was very welcome.