dougpeterson
Workshop Member
Trust me, having done tests routinely for public consumption for nearly a decade now, no matter what choices you make there are several "I can't believe you didn't" or "you should have" responses. They aren't wrong; they just show that testing is a rorschach test; you'll approach it with a bias toward whatever shooting style, technical needs, and personal preferences.I'm surprised that the 50 1.2L was used for the Canon. If has known focus drift.
If I had used an Otus on the Canon someone would have said "you should have used a Canon brand lens since that's what most people shoot with, and the Otus isn't even autofocus" etc etc. If I had used the 50/1.4 people would have said "why use the cheaper/lower-end 50". If I used used the sigma, etc etc.
Given that we can't run every sub-test with every camera with every lens (with every kind of subject matter with every kind of lighting), the results will always exclude potentially useful combinations. I think comparative tests are always very limited in nature; real world photography isn't a function of controlling every variable or matching focal lengths, it is about getting the most out of a particular set of gear, and picking the right tool for the job, and learning the gear well enough (and having gear with the appropriate features and interface) that the gear is the last thing you're worried about at the time you want to create an image.
We provide these test because we are so frequently asked for them by our customers. But ultimately I think they are far less valuable than using the gear yourself in situations similar (or identical) to your actual intended use. That's why we offer rental-toward purchase, open houses, appointment-based free use of our studio, and attend workshops and trade shows (like WPPI coming up in a bit, or the Breed Workshop we were at in LA this weekend. An hour with a camera in your hand will tell you far more than an afternoon spent looking at these files regarding whether it will suit you.
All that said, I will gladly provide my time and my company will gladly open its studio and equipment vault (in LA and NYC) to anyone who wishes to make a different comparison.