Site Sponsors
Results 1 to 11 of 11

Thread: Proper way to do controlled testing?

  1. #1
    theprophe
    Guest

    Proper way to do controlled testing?

    Hi, i currently have a gh1, and just got a nex5, i would like to do some controlled tests with different lens combos, any hints? For instance to compare a pen f 40mm lens on both systems, would i just move the tripod back when using the gh1 to get the same FOV? I was thinking of just photographing newspaper on the wall, thanks for anyhelp forth coming

  2. #2
    Senior Member
    Join Date
    Jun 2009
    Location
    San Diego, CA
    Posts
    776
    Post Thanks / Like

    Re: Proper way to do controlled testing?

    Generally you'd be correct in wanting to maintain a consistent FOV so you're looking at the same exact image on both systems, then cropping the same sections from each system's photos (center and corners, for example) to do your pixel-peeping comparison. Make sure your lighting is consistent so you have the same exposure value on both cameras, and you probably want to process in RAW to eliminate compression issues unless that's part of your comparison testing.
    -Dragos
    Panasonic GH1/G1, Canon FTb(n)/F-1, Mamiya C330F/RB67 Pro SD, Chamonix 45N-2, Nikon F5 + Assorted Lenses

  3. #3
    Senior Member
    Join Date
    Jan 2010
    Posts
    519
    Post Thanks / Like

    Re: Proper way to do controlled testing?

    It would make sense to make the comparison in MANUAL mode - set the same aperture and shutter speed. The goal is to receive similar exposure (brightness, histogram).

    It means - that both cameras may need different ISOs to achieve that.
    And this is normal, as LUMIXes need lower ISOs than others in the same light conditions.

  4. #4
    theprophe
    Guest

    Re: Proper way to do controlled testing?

    thanks for the tips, but wouldn't it be better to be on sutter priority to try and get the same iso and aperature, since there will be no movement?

    waiting for adapters

  5. #5
    Senior Member
    Join Date
    Jun 2009
    Location
    San Diego, CA
    Posts
    776
    Post Thanks / Like

    Re: Proper way to do controlled testing?

    Quote Originally Posted by theprophe View Post
    thanks for the tips, but wouldn't it be better to be on sutter priority to try and get the same iso and aperature, since there will be no movement?

    waiting for adapters
    No, because different aperture settings affect your lens sharpness, which is what it sounds like you want to test for. Aperture IS your control parameter, so you want to compare images at like-aperture, hence aperture priority.
    -Dragos
    Panasonic GH1/G1, Canon FTb(n)/F-1, Mamiya C330F/RB67 Pro SD, Chamonix 45N-2, Nikon F5 + Assorted Lenses

  6. #6
    theprophe
    Guest

    Re: Proper way to do controlled testing?

    Ya, sorry, i meant aperature priority, thanks for correcting me

  7. #7
    Senior Member
    Join Date
    Aug 2009
    Location
    Socorro, NM
    Posts
    404
    Post Thanks / Like

    Re: Proper way to do controlled testing?

    Well, it's best to be done in Manual mode. Even with aperture priority, different exposure is possible which might cause some problems in the decision making. Find out the optimum exposure setting in A mode, then dial the same into M mode for both cameras. It would depend on what kind of target are you testing. For infinity test shot, it's easier to shoot a very far way scenery (infinity focus). This type of test shot is less critical with the DOF and maintaining the perfect angle. With medium to close focus, you have to make sure you shoot the test target straight on, and remember to do bracket focus and pick out the sharpest center. It's better to have a test chart, but for a quick test, a newspaper filled with words would be fine.

  8. #8
    Senior Member
    Join Date
    Jan 2010
    Posts
    519
    Post Thanks / Like

    Re: Proper way to do controlled testing?

    Quote Originally Posted by theprophe View Post
    to try and get the same iso and aperature
    Absolutely NOT. Check what different cameras will give you in terms of IQ in the same light conditions.
    It means, LUMIX and NEX may use different ISO. If you would force the same aperture, shutter and ISO - then you will receive different brightness - so it doesn't make sense.

    Do you understand what I mean?
    Do you know, that LUMIX ISO 800 <> NEX ISO 800?

    Comparing different cameras on the same ISO - today has less and less sense.

    Have a look here:
    http://www.dpreview.com/reviews/panasonicgf1/page16.asp

    and here:
    http://www.dxomark.com/index.php/en/.../Lumix-DMC-GH1
    (section "ISO Sensitivity")

    Can you see, that if you set on LUMIX ISO 800 - than you can compare it to other cameras (if their ISO is alligned to standard) ISO 1126 (DxO) or ISO 1000 (DP Review)?

    Another example. I met once with a colleague to test VIDEO. I with GH1, him with 5D II. He asks me to set ISO 1600. I asked why? He answered - to see what noise will we see on the same ISO. I answered why you want to compare this way? He answered - let's see what will we simply get in this evening conditions. I told him, that it doesn't make any sense, as my camera will set ISO 1000 - 1200, when 5D II will need ISO 1600. If I would set 1600 too - my image would be too bright.

    He was totally confused, as many people still. Than we confirmed that in reality.

  9. #9
    theprophe
    Guest

    Re: Proper way to do controlled testing?

    yes your right Jerry, although when i went to test, im sure i would of seen the problem with exposure.

    I really thought i would get my adapters today, darn.. didn't come

    hiepphotog.. thanks for the tip on focus bracketing, didn't think of that

  10. #10
    Senior Member pellicle's Avatar
    Join Date
    May 2009
    Location
    Southport, Australia
    Posts
    1,429
    Post Thanks / Like

    Re: Proper way to do controlled testing?

    Hi

    Quote Originally Posted by theprophe View Post
    Hi, i currently have a gh1, and just got a nex5, i would like to do some controlled tests with different lens combos, any hints?
    document what you do to the nth degree. Use manual so that you do not get micro aperture or shutter changes (like 10th of a stop or so) which can effect your exposure comparisons. Set ISO manually.

    Write your comparisons, publish them and see what comments you get. Use this in a feedback loop to assist your strategy and guide what you do in the next round of testing.

    learning by doing :-)

    please publish here, perhaps with a link to a blog, these are often free and also provide image hosting and you retain control over the materials. This will also make the pages easier to load. I for one will be interested to see what you've done.

    try to avoid posting 16 images into a page and say "see what I mean" without making it clear.

    for instance:

    http://cjeastwd.blogspot.com/2009/08...two-teles.html

    is what I consider to be a basic look at two lenses. It of course lacks statistical validity (only one sample of each lens), but I'm willing to accept that the tolerances of manufacture are close enough to make each a reasonably representative sample of each lens.

    test with something in mind, with some plan.

    I agree that testing for same angle of view is desireable, but is problematic because of the differing formats. I tend towards preferring the 3:2 format to the 4:3 ... but then that's how I like to compose. Clearly this makes direct comparisons problematic.

    my expectation is you'll find that both cameras are so close in performance as to have the results effected by the changes in cropping caused by "matching for height of frame" vs "matching for width of frame" in composition of your test image.

    in some situations I prefer to use flash for my test light. I use a manual Metz flash on manual exposure. In combination with manual lenses (preferably the exact same lens this takes all things except sensor out of the equation) because light only comes from the flash, which having a duration of about 1/5000th of a second will mean that the shutter variation has no determination of exposure.

    :-)

  11. #11
    theprophe
    Guest

    Re: Proper way to do controlled testing?

    I will just post them here, i will also use the 3:2 ratio of the gh1, since it can crop the sensor properly, will make the comparison easier. I'm sure i will make mistakes. As long as there is no obvious mistakes, should be ok

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •