The GetDPI Photography Forum

Great to see you here. Join our insightful photographic forum today and start tapping into a huge wealth of photographic knowledge. Completing our simple registration process will allow you to gain access to exclusive content, add your own topics and posts, share your work and connect with other members through your own private inbox! And don’t forget to say hi!

Proper way to do controlled testing?

T

theprophe

Guest
Hi, i currently have a gh1, and just got a nex5, i would like to do some controlled tests with different lens combos, any hints? For instance to compare a pen f 40mm lens on both systems, would i just move the tripod back when using the gh1 to get the same FOV? I was thinking of just photographing newspaper on the wall, thanks for anyhelp forth coming
 

photoSmart42

New member
Generally you'd be correct in wanting to maintain a consistent FOV so you're looking at the same exact image on both systems, then cropping the same sections from each system's photos (center and corners, for example) to do your pixel-peeping comparison. Make sure your lighting is consistent so you have the same exposure value on both cameras, and you probably want to process in RAW to eliminate compression issues unless that's part of your comparison testing.
 

Jerry_R

New member
It would make sense to make the comparison in MANUAL mode - set the same aperture and shutter speed. The goal is to receive similar exposure (brightness, histogram).

It means - that both cameras may need different ISOs to achieve that.
And this is normal, as LUMIXes need lower ISOs than others in the same light conditions.
 
T

theprophe

Guest
thanks for the tips, but wouldn't it be better to be on sutter priority to try and get the same iso and aperature, since there will be no movement?

waiting for adapters
 

photoSmart42

New member
thanks for the tips, but wouldn't it be better to be on sutter priority to try and get the same iso and aperature, since there will be no movement?

waiting for adapters
No, because different aperture settings affect your lens sharpness, which is what it sounds like you want to test for. Aperture IS your control parameter, so you want to compare images at like-aperture, hence aperture priority.
 
T

theprophe

Guest
Ya, sorry, i meant aperature priority, thanks for correcting me
 
Well, it's best to be done in Manual mode. Even with aperture priority, different exposure is possible which might cause some problems in the decision making. Find out the optimum exposure setting in A mode, then dial the same into M mode for both cameras. It would depend on what kind of target are you testing. For infinity test shot, it's easier to shoot a very far way scenery (infinity focus). This type of test shot is less critical with the DOF and maintaining the perfect angle. With medium to close focus, you have to make sure you shoot the test target straight on, and remember to do bracket focus and pick out the sharpest center. It's better to have a test chart, but for a quick test, a newspaper filled with words would be fine.
 

Jerry_R

New member
to try and get the same iso and aperature
Absolutely NOT. Check what different cameras will give you in terms of IQ in the same light conditions.
It means, LUMIX and NEX may use different ISO. If you would force the same aperture, shutter and ISO - then you will receive different brightness - so it doesn't make sense.

Do you understand what I mean?
Do you know, that LUMIX ISO 800 <> NEX ISO 800?

Comparing different cameras on the same ISO - today has less and less sense.

Have a look here:
http://www.dpreview.com/reviews/panasonicgf1/page16.asp

and here:
http://www.dxomark.com/index.php/en/Camera-Sensor/All-tested-sensors/Panasonic/Lumix-DMC-GH1
(section "ISO Sensitivity")

Can you see, that if you set on LUMIX ISO 800 - than you can compare it to other cameras (if their ISO is alligned to standard) ISO 1126 (DxO) or ISO 1000 (DP Review)?

Another example. I met once with a colleague to test VIDEO. I with GH1, him with 5D II. He asks me to set ISO 1600. I asked why? He answered - to see what noise will we see on the same ISO. I answered why you want to compare this way? He answered - let's see what will we simply get in this evening conditions. I told him, that it doesn't make any sense, as my camera will set ISO 1000 - 1200, when 5D II will need ISO 1600. If I would set 1600 too - my image would be too bright.

He was totally confused, as many people still. Than we confirmed that in reality.
 
T

theprophe

Guest
yes your right Jerry, although when i went to test, im sure i would of seen the problem with exposure.

I really thought i would get my adapters today, darn.. didn't come

hiepphotog.. thanks for the tip on focus bracketing, didn't think of that
 

pellicle

New member
Hi

Hi, i currently have a gh1, and just got a nex5, i would like to do some controlled tests with different lens combos, any hints?
document what you do to the nth degree. Use manual so that you do not get micro aperture or shutter changes (like 10th of a stop or so) which can effect your exposure comparisons. Set ISO manually.

Write your comparisons, publish them and see what comments you get. Use this in a feedback loop to assist your strategy and guide what you do in the next round of testing.

learning by doing :)

please publish here, perhaps with a link to a blog, these are often free and also provide image hosting and you retain control over the materials. This will also make the pages easier to load. I for one will be interested to see what you've done.

try to avoid posting 16 images into a page and say "see what I mean" without making it clear.

for instance:

http://cjeastwd.blogspot.com/2009/08/tale-of-two-teles.html

is what I consider to be a basic look at two lenses. It of course lacks statistical validity (only one sample of each lens), but I'm willing to accept that the tolerances of manufacture are close enough to make each a reasonably representative sample of each lens.

test with something in mind, with some plan.

I agree that testing for same angle of view is desireable, but is problematic because of the differing formats. I tend towards preferring the 3:2 format to the 4:3 ... but then that's how I like to compose. Clearly this makes direct comparisons problematic.

my expectation is you'll find that both cameras are so close in performance as to have the results effected by the changes in cropping caused by "matching for height of frame" vs "matching for width of frame" in composition of your test image.

in some situations I prefer to use flash for my test light. I use a manual Metz flash on manual exposure. In combination with manual lenses (preferably the exact same lens this takes all things except sensor out of the equation) because light only comes from the flash, which having a duration of about 1/5000th of a second will mean that the shutter variation has no determination of exposure.

:)
 
T

theprophe

Guest
I will just post them here, i will also use the 3:2 ratio of the gh1, since it can crop the sensor properly, will make the comparison easier. I'm sure i will make mistakes. As long as there is no obvious mistakes, should be ok
 
Top