Does not surprise me! I always had a feeling that the very much praised Zeiss lenses made for Nikon, Canon and Sony are not really as great. maybe the build quality, but not the optical quality. One of the reasons I sold my Sony Alpha because the Zeiss zooms did not hold up against the Nikkor zooms (2.8/14-24 and 2.8/24-70).
This is not only based on my findings, but also on those of some of my photographing friends.
Nikon does make excellent glass, no doubt. However, one should take these tests for what they are. A lens that may produce outstanding results at infinity may not be well corrected at close distances. DXO tests are done at very close distances and reflect the lens performance for that particular use.
From the lenses shown above, I own the planar 50/1.4 and have extensive experience with it. I looked at the resolution tests and field map part in the DXO charts and I have no idea where did the results come from. I think they must have a problem with manual focusing. Otherwise I really can't explain the results ???
M262 ZM 25/2.8 35/1.4 50/2 85/2
If a photograph were just numbers...
I have most Zeiss lenses as well as the Nikon counterparts. The DxO results are not in-line with what I am getting as results using these tools on a regular basis. Lets put it that way.
I wonder how they have performed their tests or how they came to their conclusions? It is just that I often find them odd.
Last edited by Dustbak; 14th February 2011 at 00:55.
I'm reminded of the days of THD numbers for amplifiers and the complete lack of correlation to sound quality. The numbers don't necessarily equate to photographic ability unfortunately.
Like others here, I've owned the Zeiss and currently still own the Nikon glass. There is a distinctly different look to the Zeiss images, particularly in relation to apparent sharpness (micro contrast) and 3D separation between in focus and out of focus areas that the Nikon lenses don't achieve and I'm sure can't be measured by a DXO test.
That said, I'm very happy with my Nikon AF glass. If I were building a MF outfit for landscapes etc with a D3x again then I'd probably stick to the Zeiss offerings.
Lies, Damned Lies and statistics!
I'm really pleased (and of course I completely agree) with Dxo findings with respect to the sensor on the Pentax K5 . . . . however, these findings on lenses are clearly garbage
Where's the values for 'Feel', 'Bokeh', 'Micro Contrast', 'General Loveliness'?
Just this guy you know
Happily, there is much more to Photography than tests, reviews, statistics and bit-banging.
..."Equipment often gets in the way of Photography." ...
Godfrey - GDGPhoto Flickr Stream
I am very disappointed with DXO's lens tests. I thought they should know better, being such a professional company, that handles sensor tests in a very objective manner by the way. In fact most the results they show look fishy, and yes, numbers do lie. It all depends on the methodology and DXO seems to have the wrong methodology. A real testing bench from Carl Zeiss that measures lens performance with white light at infinity regardless of the camera sensor only costs around 200,000$. I think a company like DXO can afford one easily. What they do however is stick a testing chart on the wall, put the camera on a tripod, focus the lens, and take a photo that they analyze with their software. The first drawback of this method is that the biggest testing target available is 80x120 cm if I remember correctly. For some WA lenses this would mean less than 50cm shooting distance and for most lenses, they would be shot at MFD. Of course there are other issues like accurate focusing, camera alignment, lens alignment, AA filters and what not. The list of things that can go wrong in quite extensive.
M262 ZM 25/2.8 35/1.4 50/2 85/2
If it is true that DxO takes images from sample charts and examines them with their software that will be one of the most flawed ways to test lenses. They basically test the whole system including their own eyes and focussing skills. You cannot say which was the determining factor in the outcome.
MTF charts are in that case much more reliable and even with MTF charts you can have calculated charts or measured charts. In the MTF charts provided by Zeiss for their lenses they use measured charts.
Zeiss's numbers show different things than DxO.
If numbers don't lie how is that possible?
I much rather rely on Zeiss's numbers than on DxO's if I have to make a choice based on numbers. In the end I trust only myself and leave the numbers for what they are.
Last edited by Dustbak; 14th February 2011 at 23:55.
As a one time CEO in a former life, I well remember asking my financial director (VP in the States) what profit we were likely to make for that year, his reply was "how much do you want to make".
The only reviews I trust are from non aligned working photographers who use the equipment every day to please customers and make themselves a living.
Fan club members of this or that camera or lens I ignore for usually what they are! Similarly magazines who carry a lot of advertising for usually just two brands!
Exactly! There are plenty of real life samples on various forums, notably FM, that show what these lenses are capable of. Numbers are meaningless, especially if they contradict the real life results.
M262 ZM 25/2.8 35/1.4 50/2 85/2
What are we trying to prove here.
For some Nikon is better.
For some Zeiss is better.
What is better and works for you.
Some of my friends find Sigma works wonders. For others it is Tamron.
I find Zeiss works for me. I like what they produce for me. My posts are from Zeiss.
You might not like them. I like the results.
Let's take more photographs and show what we can do. Irrespective what lens we used to create our vision.
I just love those test done under control conditions, I under stand that you must set a bench mark.
how many times do you shot with full control of all the elements?
how many times do you shot at the widest aperture on the lens?
My point is simple, every top lenses maker that make (Pro Prime Lenses) make fine optics that will perform well at a certain range of apertures and conditions. The best test you can do is to actually get the lens go out and shot, shot the way you like example: your range of f-stops, prefer type of lighting conditions and focus points.
If possible try renting the different lenses you are considering before buying and shot with them. You will see that certain lenses will work for you and other won't.
The good news is that both Zeiss and Nikon are offering some excellent lenses. You may decide the extra expense of a particular Zeiss lens makes sense in order to obtain a certain look. But if—for budget reasons for example—you buy the Nikon instead, it's not like you're getting a complete dog. I think we're fortunate to live with so many great options for gear.
Had a talk with DxOMark.
They said the lenses could be 'sends in' from people, or they could be 'bad' examples.
They are waiting on lenses/info/stuff from Leica.
But DxOMark does make money from their software, so they do want to be accurate (and at the same time, they could create drama to bring people to their site).
M262 ZM 25/2.8 35/1.4 50/2 85/2
I agree, taking test charts and symplifying the results is misleading.
But their camera tests came up with some misleading results as well (like cameras with the very same sensor and similar real-wold results performing entirely different).
But Nikon is capable of designing and making fine lenses and the "Carl Zeiss"-lenses shown here are made by Cosina and designed to meet their capabilities in manufacturing and Zeiss' profit request. Carl Zeiss only manufacturers very few photographic lenses, their professional lenses are decades ahead in optical and mechanical quality... and price ;-)
We've learned to love brands, not manufacturers and Zeiss knows that - putting a Zeiss-design on Zeiss-testing-equipment doesn't make it a Zeiss-lens, not for this price.
Zeiss lenses have been made in Japan for ages. This doesn't lessen their value in any way. I'm sorry, but if you look at DXO's resolution graphs of the Distagon 21, a lens most famous for its sharpness across the frame at all apertures, you would be very shocked. This alone takes all credibility from these tests. This actually borders on the ridiculous. I want some of the stuff they're smoking.
M262 ZM 25/2.8 35/1.4 50/2 85/2
I know from my own testing of optics (and am certain most others here know too) that most lenses if tested at three major distance ranges such as MFD, mid distance and infinity, generally perform weakest at MFD. Not all lenses (such as specifically designed macro/micro) but many do. Recently I tested out a couple of wide angle lenses and a telephoto and their performance at MFD was downright mediocre, but at mid and long range with their remarkable performance, one wouldn't believe these were the same lenses. So if DXO is testing off a printed test chart at near MFD range, then even two similarly performing lenses at most distances can look and perform quite differently at MFD. Just something to consider. It's not that DXo is rigging the tests, it's apparent their methodology is weak. I wish testing and subsequent results were gives for different camera/lens to subject distances, which would be a more reliable indicator of a lens performance, at least resolution wise.
I have a degree in business management/accounting, so i know all about numbers and how they can be misleading if their source is faulty/wrong/etc (i believe in English its called falsifying the books).
I was generally talking, the same as when i would say 2+2=4, or when tests are done in a control environment, you can assume the results are good for what you/they want.
But like i stated, numbers only say so much... ESPECIALLY in lenses!
Newer designs engineered and manufactured by Zeiss (with exception of the Compact Primes, which are just ZF-designs) incorporate more advanced designs and technologies with tighter tolerances - that makes them so expensive. They just introduced a f1.2 lens with 90° horizontal angle of view for Super35 - it makes use of radical aspherical elements that cannot by handled by Cosina or Sony. The Distagon 21 is "boring" stuff for them...
When those lenses would test mediocre against "regular" Nikons it would be suspicious, but DxOMark might just got bad samples from Cosina. Sample variation is another issue, somebody who claims comprehensive testing should buy several samples and include the variation in the results.
No, some might have said this but not the majority. Actually the chances of getting a Zeiss lens with sample variation ought to be much smaller than with Nikon. Part of what you are paying for is the additional QC. Sample variation would be a very bad excuse for these 'remarkable' numbers.
Read the calculated MTF charts of Nikon & the measured MTF charts of Zeiss and see the numbers that are derived from tests that IMO have a lot more authority. The MTF charts paint a totally different view than DxO making me severely doubt their practices with regard to lens testing.
On the Zeiss site you need to click on the link datasheets.
BTW, using different interpretations or systems that lead to different outcomes is not falsifying the books, not as long as you are staying within the boundaries of the law that apply in that particular case. Naturally interpretations or usage of systems can be open for discussion or court rulings but still no falsifying perse. I don't have a degree in accounting but used to be a tax advisor in a previous life which might count for my different view
Quote from DxOMark's website:
"Although we can never exclude outliers, our tests show that it is not very likely that variances in lens manufacturing will significantly change (for better or worse) the performance of a lens."
[ The quote is from this page: http://www.dxomark.com/index.php/Lea...ty-and-DxOMark ]
This does not coincide with my personal experiences.
And it doesn't coincide with all the totally opposite statements from different people concerning a specific lens model or a specific camera model.
I firmly believe in sample variation, not only with regards to lenses, but also with regards to cameras.
I have come to a point where I only trust what I can see with my own eyes.
I therefore wish that more forum members, reviewers, test labs etc. would link their tiny webified sample images to full resolution versions so that we could see for ourselves and judge for ourselves. (Actually reviewers and test labs ought to link to the RAW files as well).
My own observation WRT Nikon and Zeiss lenses (for Nikon) were that the Zeiss lenses are in best case equal to Nikkors, in many cases inferior. And of course they do not have AF. So end of the day what remains from the Zeiss glass for me is that I can say I have taken a specific shot with a Zeiss lens - I can tell you I do not give ..... on this any longer.
My suggestion - try to test the lenses you want to buy for yourself and compare, this is the best way to go!
My version of a test.
A few years ago I was considering the purchase of the Nikkor 105 f2.8 AF to replace my Micro Nikkor 55 f 3.5 AI, mainly for studio close up shots of the product. (AF was not a must in this case, just to get newer lens design construction)
The set was backpacks on white background, since the client wanted some close-up on details a very short focus lens was needed. I keep my background 1 to 1.5 stop over the product for close cutting ease, so light fringing is a factor and it plays on the contrast, a prime lens coating and shades are needed to control that fringing.
So I rented the 105 Nikkor from my local shop and shot some with the 55 and the 105 of the same details.
Under my needs and condition for that type of lens. Only then I was able to say if that lens versus the other was worth the investment.
To me that is the best test you can do to compare one to the other.
so true, Doug
Tax Advisor... blah I still remember my tax/law courses (both Canadians and US). I much preferred the administration side of business than financial.
I do find it interesting though, that people are complaining/arguing because the results aren't in line with their believes of 'legendary' Zeiss lenses.
Anyone considered that those testes were correct?
Now i'm not saying anyone is wrong, or gaga, if it works for you, thats the ONLY thing that matters, but still... if the results were the other way around... this conversation would be quite different
This statistical analysis works for me - as a work of fiction, that is.
So, apt that it is sourced from some place called Nikon 'rumours'! Indeed.
Where is their 'test' of microcontrast or of contour definition, two of the crucial characteristics that make images worth looking at?
It appears they do not have one...how about modulation across the frame, or anything more than a *single figure* for resolution? Is that not just a little too abbreviated? How about Zones A,B and C maybe? No.
It looks like informed people will just have to make do with Zeiss's industry leading MTF charts, which accurately depict continuous MTF across the frame for 10,20 and 40 lpmm, at typical taking apertures: wide open and two stops down, for infinity (standard measure) and for image ratios of 1:10 and 1:5 (or 1:2), for *actual* lenses. With distortion and light fall-off data across the frame thrown in.
Where can one find equivalent data for the (here) vaunted Nikkor lenses? Well, we cannot - because they are not made available by Nikon, who would certainly have access to Zeiss's industry-leading optical bench. Sad, but true.
FWIW, the two Makro Planars are very close to peerless image producers, in *any* company, excluding perhaps the Leica APO 100/2.8, for the longer FL lens.
It seems you are stating that DxOMark results are false, which we are still waiting to be proved (that their results are wrong).
Personally, i think the result are in line from what iv been told by Zeiss users. Crappy lab performance, outstanding real-world results.
What is wrong with this picture:
1) Zeiss's Lab tests, which are well documented and accepted for their scientific value : Excellent.
2) Field reports/experience by users (including myself) which are admittedly subjective : Excellent.
3) DxO, while their test methodology and its scientific value is yet unknown: Mediocre at best.
"Numbers don't lie." Oh, really?
On my accountant's office wall: "Numbers don't lie, but they speak many languages, and can be made to say many things".
Anyone who says 'numbers don't lie' has never done any logic, or deeper maths, for sure. Numerical results—while perhaps internally consistent— depend on starting assumptions. The DxO results equilibrate quite different qualities (qualia in philosophy-speak)—the assumption is that these qualities can be arrayed and compared as if they are the same kind of things (which a moment's reflection will show that they are not).
Although seemingly less 'scientific', qualitative assessments are often much closer to real-world experience (and, from my perspective, big "R" reality is never wrong!).
The DxO tests are flawed, in this exact way. HTH, KL
I've owned the Nikon 85/1.4 but not the Zeiss and the Zeiss 50/2 but not the Nikon.
The only corresponding pair I've owned was the Nikon 35/2 and the Zeiss 35/2, and I can tell you that in that instance, the Zeiss was dramatically better in nearly every way except autofocus speed .
Lens reviews need images. Measurements aren't enough!
Or i can just tell you, numbers don't lie, in this case (we aren't talking about something else, lets stay on topic), since DxO tests under controlled environment. Or else what would the point be? Unless you are saying they are deliberately doing this (see other response, creating drama).
BTW if you read another response, you would see that they were contacted and said it could be bad examples of lenses (which started another conversation).
DxOMark is still waiting on Zeiss to send them new examples.
AFAIK no one buys lenses base on a couple measurements only.
This thread caused me to google Zeiss lenses and the first article that popped up was this, which echos some of the sentiments posted here.
“DxOMark focuses only on image quality performance but does not cover many other criteria that are important when looking for cameras or lenses that fit your needs.”
So, they do say you need to tread carefully through their pages. More, they also state that “In particular, DxOMark does not measure or consider such criteria as mechanical robustness, shooting time and rate, ease of use, back LCD display quality, camera control ergonomics, flexibility, value for money, etc. While sensor and lens performance are critically important, they are not the only factors that should be taken into consideration when choosing a digital camera.
Also, with its current scope, DxOMark does not measure the quality of the image processing, nor do we address “camera control” (sometimes called “3A” for Auto Exposure, Auto-Focus, and Auto White Balance).
To give a broader perspective to our readers about lenses and cameras overall performance, DxOMark provides links to articles, reviews, and analyses of photographic equipment produced by renowned websites, magazines and blogs.”
I never heard of DxOMark before a few minutes ago, but it seems that their tests are limited and inform the reader that their results are not exhaustive and do not take into consideration factors that many photographers may find important in choices of equipment.
Someone made an audio analogy and I'll make a similar one. Transistors revolutionized the audio industry and others, but all the numbers in the world couldn't pry tube amps from the hands of those who loved them for the warmth of their output and their resistance to clipping.
As for THD, the human ear can tolerate about 10% THD, so when the THD wars were in their heyday and THD figures were out to the thousandths of a percent, an apparent huge difference would be completely indistinguishable to the ear.
I'm new to photography, so I'm learning a lot here, but the fact of the matter seems to be when it comes to art and technology, some of us prefer some flaws to others and distortion is a matter of taste and is desirable in some cases.
Oh, and hello to everyone. I am GradyPhilpott.
There is nothing subjective about mtf numbers, they are as objective and scientific as can be. The mtf numbers at DXO are clearly incorrect, and for sure not due to decentering as it is shown clearly in the field maps. My guess is their bad manual focusing abilities, combined to shooting at MFD lenses that are designed for infinity.
M262 ZM 25/2.8 35/1.4 50/2 85/2
All i'm saying hear is that test methods are key in stereo and they are in photography as well. The tests performed by DXO are simply not adequate to predict the performance of the device under test in the real world. Someday we may have a universally agreed upon standard by which to judge the comparative performance of lenses..............but not today. So let your eyes be the final arbiter.
To those that say they have had both the Nikon and Zeiss lenses and found
Nikon to be better.
Good for you.
I have only purchased one sample each of the zf 25/2, 35/2, 50/2, 50/1.4 and the 100/2 version 1.
I used to have a big collection of Nikon lenses.
Now I only use Zeiss.
Good for me.
I have not printed poster size..I do not intend to. I print as wall hangers.
I post in this forum. It is there for all to see. Good or bad.
I am happy with my Zeiss results.
Numbers won't convince me. My prints do.