Lens testing in general is a nightmare, and why I stopped posting my own results several years back. No matter how you structure your test, you will have great difficulty looking at multiple variables in a single test. THus it takes several controlled comparisons and most of us don't really want to spend that kind of time performing them.
Here is what I have gleaned from years of doing lens tests:
1) Resolution testing. The ONLY way you can compare resolution is to shoot resolution targets that generate empirical evidence of actual lines/mm resolved. You can have situations where lens A looks softer than lens B across the image, yet lens A discriminates more lines/mm than B. Better if the test chart has resolution grids in the three RGB color channels in addition to black, and has them displayed vertically and sagittally as well as horizontally. (Here is where a real version of the USAF optical test target can help a lot.) You will be surprised at how many lenses render some colors better than others or how different an individual lens can be on sagittal resolution relative to horizontal...
2) Distortion testing. A NIGHTMARE! I used to use a building in my neighborhood that had lots of vertical and horizontal structural components like windows and beams --- NOT bricks! With this, I got far enough away in the parking lot, then leveled and squared the camera as it is easier to evaluate from a distance where being off becomes more visible and thus avoidable. Mark your position in the parking lot for future tests.
3) Then you have to triple check that you have the proper lens matched to the image --- easy with EXIF, but when testing a bunch of different third party lenses via adapters, it can get arduous to keep every thing organized.
Corollary to 1) And from my experience, you will almost always choose to own lens B in the above situation #1 even though it registers less resolution, so spending time on the rest of the testing is pointless anyway
What I do now is simpler: I take each lens under consideration and shoot normally with it, exposing various subjects of various colors at various distances and various apertures. I then scrutinize the images initially and make my final decisions based on my partially empirical yet often mostly SUBJECTIVE experiential appraisal of what I see on my monitor: if I like what I see I keep the lens, if I don't I return it.
Now to the discussion in the above posts, via a PS on corner resolution testing on wideangles: It is almost always an exercise in futility due to one simple optical phenomenon: curvature of field. An example is the 28/f1.8 Canon. I tested that lens early on, having shot it on my standard flat test target and pronounced it as horrible in the corners to the point I considered it unusable. A few years later, I obtained one as part of a bulk gear purchase. I was going to sell it immediately, but then tested it per my revised "shoot it" test as described above. What I learned was eye-opening: It was stunningly sharp in the corners, just at a distance significantly closer than where the center of the lens was focused due to significant curvature of field! (In that case, if I focused the center at 10 meters, the corners were crisp at 2 meters, even wide open.) Moreover, I discovered for my style of shooting, this was actually a positive trait: with the lens angled down slightly, foreground elements through the mid-range elements I had focused on could be held in good focus at wide open apertures -- this can a superb "characteristic" for street and landscape shooting if and when exploited, and was a significant "ah-ha!" for me. (FWIW, when I sold off the bulk of my Canon gear to help fund the medium format purchase, I kept two lenses -- a particularly good example of the 50/1.4 and that 28/1.8.)
Finally, please note that curvature of field and rectilinear distortion are almost always battling elements of wideangle lens design -- you cannot have both, so lens designers have to choose one in favor of the other; if the design favors low distortion, it will likely have higher curvature of field; if the design favors flatness of field, it will likely have greater linear distortion.
Cheers,
Here is what I have gleaned from years of doing lens tests:
1) Resolution testing. The ONLY way you can compare resolution is to shoot resolution targets that generate empirical evidence of actual lines/mm resolved. You can have situations where lens A looks softer than lens B across the image, yet lens A discriminates more lines/mm than B. Better if the test chart has resolution grids in the three RGB color channels in addition to black, and has them displayed vertically and sagittally as well as horizontally. (Here is where a real version of the USAF optical test target can help a lot.) You will be surprised at how many lenses render some colors better than others or how different an individual lens can be on sagittal resolution relative to horizontal...
2) Distortion testing. A NIGHTMARE! I used to use a building in my neighborhood that had lots of vertical and horizontal structural components like windows and beams --- NOT bricks! With this, I got far enough away in the parking lot, then leveled and squared the camera as it is easier to evaluate from a distance where being off becomes more visible and thus avoidable. Mark your position in the parking lot for future tests.
3) Then you have to triple check that you have the proper lens matched to the image --- easy with EXIF, but when testing a bunch of different third party lenses via adapters, it can get arduous to keep every thing organized.
Corollary to 1) And from my experience, you will almost always choose to own lens B in the above situation #1 even though it registers less resolution, so spending time on the rest of the testing is pointless anyway
What I do now is simpler: I take each lens under consideration and shoot normally with it, exposing various subjects of various colors at various distances and various apertures. I then scrutinize the images initially and make my final decisions based on my partially empirical yet often mostly SUBJECTIVE experiential appraisal of what I see on my monitor: if I like what I see I keep the lens, if I don't I return it. Now to the discussion in the above posts, via a PS on corner resolution testing on wideangles: It is almost always an exercise in futility due to one simple optical phenomenon: curvature of field. An example is the 28/f1.8 Canon. I tested that lens early on, having shot it on my standard flat test target and pronounced it as horrible in the corners to the point I considered it unusable. A few years later, I obtained one as part of a bulk gear purchase. I was going to sell it immediately, but then tested it per my revised "shoot it" test as described above. What I learned was eye-opening: It was stunningly sharp in the corners, just at a distance significantly closer than where the center of the lens was focused due to significant curvature of field! (In that case, if I focused the center at 10 meters, the corners were crisp at 2 meters, even wide open.) Moreover, I discovered for my style of shooting, this was actually a positive trait: with the lens angled down slightly, foreground elements through the mid-range elements I had focused on could be held in good focus at wide open apertures -- this can a superb "characteristic" for street and landscape shooting if and when exploited, and was a significant "ah-ha!" for me. (FWIW, when I sold off the bulk of my Canon gear to help fund the medium format purchase, I kept two lenses -- a particularly good example of the 50/1.4 and that 28/1.8
.)Finally, please note that curvature of field and rectilinear distortion are almost always battling elements of wideangle lens design -- you cannot have both, so lens designers have to choose one in favor of the other; if the design favors low distortion, it will likely have higher curvature of field; if the design favors flatness of field, it will likely have greater linear distortion.
Cheers,
