IQ4150 Frame Averaging Comparisons

[dougpeterson]

This pair of samples compares one 8-second exposure where I walked through the scene holding a flashlight (pointed at the camera) and a frame average of 8 1-second exposures with ISO adjusted to achieve the same overall exposure (ISO 50 and ISO 400).

There are two things of note:

1 - in the single frame the light streak is full exposed at 100% luminosity (RGB 255,255,255) and is completely opaque. In the stacked frame the light streak is not fully exposed (around 85% luminosity) and you can see some detail behind the light. That's due to the averaging nature of having one frame with the light and 7 frames without the light.

2 - Notice that in the averaged version the light streak is pink/magenta rather than white or gray. I've been working with my dealer to create a support case for what appears to be a problem in the calculations. Anytime a bright highlight is only present in a very small percentage of the frames it's rendered pink/magenta rather than gray. A similar stack in Photoshop would render it as gray. My guess is that it has to do with how the two green channels are handled in the calculation. It doesn't seem to matter if the highlight is from an LED, tungsten or flash source. The problem is very repeatable on my back.

I think (2) is magenta in the highlights because it's blown out in some of the frames that are being averaged. It's important not to blow out pixels in a frame-averaged image. The pixels near the blow out can show purple fringing which will then average in. The same thing can happen on seculars when shooting seascapes for example.

I'd suggest using the Raw File Clipping Warning in live view to avoid this.

 

[JimKasson]

...Notice that in the averaged version the light streak is pink/magenta rather than white or gray.

That's because the averaging is being done prior to white balancing, so RGB = 1,1,1 is not white after WB.

Jim

 

[Craig Stocks]

Whatever the mechanism of the error turns out to be Phase One needs to address it. I've submitted RAW files for the support case but I doubt if we'll ever hear the details. FWIW - the Sony frame averaging tool works fine in that regard and does not pick up magenta in highlights, they come out as a nice gray.

 

[Boinger]

I think (2) is magenta in the highlights because it's blown out in some of the frames that are being averaged. It's important not to blow out pixels in a frame-averaged image. The pixels near the blow out can show purple fringing which will then average in. The same thing can happen on seculars when shooting seascapes for example.

I'd suggest using the Raw File Clipping Warning in live view to avoid this.

The explanation is actually very simple for that.

Waves are things that change colors from extreme to extreme.

What you are seeing is not an artifact but white breakwater of the wave being averaged with the dark water that is probably present in that part of the picture when the wave is not there.

It is working as intended, in that situation you would have to simply increase the frame count to account for that.

 

[Craig Stocks]

For what it's worth (if anyone's still following this thread)... I did some more test shots with the IQ4 150 to reproduce the magenta cast in intermittent highlights in a frame averaged sequence.

The short version - The green channels are the first to clip, so I presume averaging clipped green channels with un-clipped red and blue leads to a magenta cast. However, the magenta cast does not show up in a single frame where highlights are clipped in the green channel or in a frame averaged set with constant illumination. Also, given the same test process and in-camera frame averaged RAW output the Sony a7R2 does not show magenta in the highlights.

The longer version:

Test setup: Phase One XF with IQ4 150, ISO 50, 1.6 seconds @f/8 with Phase One 120 mm MF macro shooting through an LCC plate. The target was a white board illuminated by and LED source. The result is almost evenly illuminated except for some natural vignetting. I captured a single frame and a FA frame from 10 frames. I also did an FA set where I turned off the LED after the 1st frame.

All photos were viewed in Capture One with all sliders at their defaults and exported as sRGB JPEGs to share. I also exported DNG versions to analyze with RawDigger, especially single frames so I could understand what and where clipping was occuring.

Results:

The center of the single frame shows clipping in the two green channels, no clipping outside of the center.

The single frame does not show a magenta cast in the center

A stack of 10 frames with the LED illuminated throughout does not show the magenta cast, in terms of color it looks just like the single frame.

A stack of 10 where the LED was turned off after the 1st frame shows a strong magenta cast in the central region where the green channels were clipped in the single frame.

The Sony A7R2 using the same EV (ISO up one stop, shutter speed down one stop) did not clip in the green channels and did not show any magenta cast

The Sony with exposure increased one EV did clip in the green channels about like the P1 but did not exhibit the magenta cast.

I still believe it's a problem with the way P1 is doing the averaging, but it's probably the natural outcome of averaging clipped and non-clipped channels. With the greens clipped the red and blue would take over. However, a single frame with similar clipping doesn't result in a magenta cast and Sony's process seems to accommodate the clipping without introducing a color cast.

Bottom line, very bright transient highlights of any size in a frame averaged set can take on a magenta tint.

 

[dougpeterson]

Craig: that's a great contribution to the community! Thanks for all the work!

 

[Wayne Fox]

The center of the single frame shows clipping in the two green channels, no clipping outside of the center.

Did you try the test to avoid any clipping with the IQ4 150?

 

[Christopher]

It should not be necessary. As it should render as grey not pink.

Did you try the test to avoid any clipping with the IQ4 150?

 

[Craig Stocks]

Did you try the test to avoid any clipping with the IQ4 150?

Yes, I did not see the magenta cast if the green channels were not clipped.

 

[etrump]

I’ve noticed it’s better to avoid clipping in all channels with a 1/3 stop under exposure as the shadows remain clean when raising them in post. Clipped channels do not seem to be handled like C1 where the missing detail is estimated from the other two channels.

Yes, I did not see the magenta cast if the green channels were not clipped.

 

[Wayne Fox]

It should not be necessary. As it should render as grey not pink.

the sensor supplies false color data because one channel is clipped, and the camera is supposed to figure out there’s a problem?

Necessary or not, it seems to resolve the issue, and since frame averaging lowers the noise floor dramatically, dropping exposure slightly shouldn’t be an issue.

 

[Christopher]

Well Photoshop does it, Sony does it... so shouldn’t be hard for Phase One... or perhaps it is.

Besides that. There are always scenes where one cannot get around not blowing something. And it should be possible.

the sensor supplies false color data because one channel is clipped, and the camera is supposed to figure out there’s a problem?

Necessary or not, it seems to resolve the issue, and since frame averaging lowers the noise floor dramatically, dropping exposure slightly shouldn’t be an issue.

 

[SCHWARZZEIT]

Well Photoshop does it, Sony does it... so shouldn’t be hard for Phase One... or perhaps it is.

Besides that. There are always scenes where one cannot get around not blowing something. And it should be possible.

Photoshop is able to do it because the averaging is done on demosaiced and white balanced image data where the raw converter has already recovered some of the lost highlight information by adjusting the other channel and interpolating. The IQ4 Frane Averaging only works with the raw data.

Based on the strict mean stacking math the magenta cast in Craig's sample images comes from the clipping of the green channel. It's not a bug. In those areas the sensor couldn't capture the additional levels of green coming in on top of the clipped data while the other channels captured their light levels correctly. Since Craig clipped the green channel only in one of the ten averaged frames the averaged green levels in those areas in the final raw file are far from clipped. But as Wayne already mentioned, it's false information. The correct amount of green is missing. And as Jim pointed out, when the camera profile and white balance is applied the false color shows up as magenta (lack of green). Since the raw file isn't clipped Capture One doesn't do any of the highlight color recovery that it can do when a channel of the raw file is clipped. The software can't tell if this is false color or not.
Ten frames aren't enough to smooth out such a clipping in one of the individual exposures, especially since the other frames got much lower exposure with the LED panel turned off. If the light would not have been turned off but only dimmed a stop or so to bring the green back in range for the rest of the sequence then I'm sure the magenta cast would have been less significant. On Craig's other sample with the light trail trough the frame it's basically the same thing. The local areas where the false color occurs was clipped only for a single frame in the stack of ten while those areas received much less light in the other frames.

Doug already mentioned that the issue can be avoided by simply underexposing enough to avoid any clipping or by using a lot more frames for the averaging. If these conditions cannot be met then Frame Averaging might not be the right tool. For long night sequences you should be aware of your scene to protect the lens from bright lights crossing the frame that may cause clipping. In the large format days I used to hold the dark slide in front of the lens during long night exposures when I felt the image required protection from temporary bright light sources. With the Frame Averaging tool you can simply stop the sequence earlier and then continue when the light levels are back to where you want them to be. But currently the two second max shutter speed limits the use of this tool for night photography anyway.

I think the Frame Averaging tool is powerful enough that it deserves a proper description in the manual with warnings about the pitfalls that the user might face. But I'm all for user control and thus hope that the Frame Averaging tool will be expanded to longer shutter speeds and also to allow different stacking algorithms.

I guess the Sony SR app uses a different math. I haven't used it nor do I have a compatible camera. But it may have other limitations.

-Dominique

 

[etrump]

My understanding is the averaging is happening on chip with purely raw data, in real time before it is read from the chip and piped to the control system.

Substantially different process than interpolating during raw conversion then averaging.

Well Photoshop does it, Sony does it... so shouldn’t be hard for Phase One... or perhaps it is.

Besides that. There are always scenes where one cannot get around not blowing something. And it should be possible.

 

[RLB]

I think the Frame Averaging tool is powerful enough that it deserves a proper description in the manual with warnings about the pitfalls that the user might face. But I'm all for user control and thus hope that the Frame Averaging tool will be expanded to longer shutter speeds and also to allow different stacking algorithms.


-Dominique

Amen. Phase One I hope you are listening. Great tool, but need more input so we can make the most of it, and directly from Phase One, not dealers, not users.

R

 

[SrMphoto]

Well Photoshop does it, Sony does it... so shouldn’t be hard for Phase One... or perhaps it is.

Besides that. There are always scenes where one cannot get around not blowing something. And it should be possible.

Photoshop does it, but it produces a TIFF as a result not a RAW. You have to modify the raw images before merging them, and you have to manage many images instead of only one. Much less convenient in my book.

Sony does it, but only with its Playmemory app. Very cumbersome IMO, and it does not work on a7rIII, a7rIV, or a9.

Several non-Sony brands implement multiple exposures well and produce RAW images like Phase One, but they heavily limit the number of possible exposures to be averaged (4-9).