« In Defense of Depth | Main | I Should Have Said... »

Wednesday, 20 June 2012


"That's when I started firmly recommending that people should never buy a lens without return privileges, and that they test it thoroughly as soon as they got it. "

Certainly good advice.

I had to return a Canon EF-S 15-85mm lens when, on the initial tests, it showed terrible decentering. Many on the forums confirmed a serious product variability.

More recent are the Panasonic m4/3 14-42x and 45-175x lenses which have shown much variability, as confirmed by *many* on the forums.


"Recently I had the reason and the opportunity to investigate that question in Epson 3880 printers, and got some surprising results."

Is this where we find out what you did with all of the sample prints of the dockside cranes and water picture.

Consumer Reports is the only consumer testing organization I know of that actually makes an effort at learning something about quality control (partly through multiple samples, partly through a large yearly questionnaire to their readers). They also make a point of getting their test samples through normal retail distribution, rather than in letting the manufacturers fine-tune them for optimal performance.

And they do sometimes test photo equipment, though not at this kind of level of detail. Their ratings are intended for normal consumers, not for enthusiasts or professionals.

Very interesting about the Schneider Componon 50mm ƒ/2.8, a lens. I used one in the 70's and thought it was very mediocre in spite of what other photographers told me. It was probably for the reasons you pointed out in your article. Eventually switched to an El-Nikkor and was more than pleased.

So far I've tested five samples of the infamous Panasonic PZ 14-42mm X lens (pancake zoom) on my G3. EVERY sample gave me blurred pictures at shutter speeds between 1/80 and 1/200, and almost every picture was more or less affected. (Btw. stabilizer on/off and with/without tripod doesn't matter, problem is always there.)
What does these statistics tell us about Panasonic...?
The main problems are that
- the lens behaves different on different m4/3 camera models (it's a problem related to the motion of the shutter),
- some people don't know how to test a lens for problems (and/or critical sharpness).
This makes it easier for Panasonic to deny this issue.
The problem is well documented (even dpreview has it in a review about the GX1), but Panasonic does nothing and the service says it's a problem of a single lens or two. (But they can't repair the defect or they send the lens back with the statement that it's ok.)
That's my statement about quality control and sample variations in modern camera gear.

I must have led a charmed life. After having purchased and used quite a variety of photo equipment over the years I can say that I've never suffered the kind of manufacturing faults that many people report. Offhand I can only recall two such instances, and neither was a lens.

I'm either very lucky or a poor photo gear tester. Probably both, as I don't tend to spend much time analyzing products. But folks in photo forums seem to spend the majority of their free time "testing" stuff rather than using stuff. Perhaps that's a blessing in many cases.

Nevertheless, I cringe a the thought of getting a crappy 100lb printer. I've had excellent luck with my Epson 3800 and loathe the idea of having to replace it.

I don't know if you've seen Roger Cicala's articles and analysis of lens copy variation. As the owner of lens rentals, he DOES have access to a bunch of copies of certain lenses and has a lot of data.



I'm glad to hear that there follow-up to the 3880 testing is on the way. I've been curious about the outcome since I sent my sample prints in to be a part of the evaluation.

Ctein, you just opened a big can of worm!

When I started this hobby (shortly after Pangaea started to break up) I was just happy to have more than one lens for my SLR. Any lens! Lens quality or sample variation never entered my mind.

I was happy.

Now I know too much to take good pictures.
(That was a bit tongue in cheek)

But, I think, in my old age I am coming full circle. If it'll fit my camera and not fall off I'll shoot with it.

Ah yes, now I'm feeling better.

I took part in this experiment and hope to hear that my printer is above average.

I am looking to buy an Epson 3880 in order to print on metal and glass. Sadly, at the recent photo show in Seattle, the demo model's horizontal feeder was broken. So a test for feed skewering could not be done. Your review of the printer quality is of great interest.


Dear Mike,

You could be right about the S-Orthoplanars. Arthur Kramer had strongly recommended I include them in my tests; he raved about them. Problem was that not only could I not get my hands on them, I couldn't even get full technical information about them. So I don't have any first-hand knowledge as to whether they are genuine apochromats or just incredibly good lenses.

Probably this is the place to stick in a note to the effect that all those enlarging lenses with APO at the beginning of their names? Those are not apochromats. It's deceptive marketing. It's a term the lens makers decided to agree upon to indicate a lens that was better than their usual achromats. But the lenses most decidedly do not meet the definition of apochromatic. Not even close.


Dear Bill,


pax \ Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
-- Ctein's Online Gallery http://ctein.com 
-- Digital Restorations http://photo-repair.com 

Dear DDB,

One of the things Mike and I both agree on is that photographic manufacturers don't cherry pick their products for our reviews. We both have ample evidence that this is not the case, not with the numbers of defective products we get, and sometimes even the replacements for the defective products.

I suppose it's possible that if you're the 500 pound gorilla on the block like Popular Photography, there is actually someone at some of the companies who pays more attention to making sure you get a sweet unit (I doubt it, mind you, but I can't say it's impossible). But at the levels that folks like Mike and I get to play, it doesn't happen.

At least that's one less thing for us, as reviewers, to have to be worrying about. We're just not important enough to be “gamed.”


Dear David,

Oh thanks for that! I'd forgotten all about Roger's articles when I was writing this column. Yeah, he's got some good information.

A bit of perspective-putting, though. Ignoring the couple of obviously defective samples he reports on, the resolution spreads he shows are less than + or - 10% around the average. That's measurable but essentially unobservable in real-world photography. So, yes, there's a difference that he can see in his laboratory, but a real-world user, even a fussy one, would not likely notice the variance unless they happened to get both near-best and near-worst lenses to compare.


Dear Emmanuel,

OMG, I've never done anything like that before! I am in a panic. Whatever shall I do?


pax \ Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
-- Ctein's Online Gallery http://ctein.com 
-- Digital Restorations http://photo-repair.com 

Funny, I'm sitting here with a Canon EF 70-200 2.8 IS L II, bought new in march, couldnt focus consistently with the thing on my 5D II (measured it to +17 in MF adjustment!), returned to shop, they sent to canon repair, they found no flaw and have returned it to me, and here 3 months later I still cant get consistent AF on the (very expensive) thing... Next stop Canon repair with lens AND camera...

A bit of perspective-putting, though. Ignoring the couple of obviously defective samples he reports on, the resolution spreads he shows are less than + or - 10% around the average. That's measurable but essentially unobservable in real-world photography.

I think that's about what he says. Most of his lenses cluster pretty tightly, with the defective ones sticking out like sore thumbs. I seem to recall that he also found some variability with lens and body matching- lens A looks great on body A but ordinary on body B- and even from test to test with exactly the same setup. So the best lenses in his tests may not be appreciably better than average, just have gotten a favorable test.

Quick question about the math from your example, Ctein, as it's been a couple years since I took statistics:

Did you take the chance that a single unit would not be lousy (80%) and multiply that by itself 10 times to get the ~10% chance of getting 10 good units in a row?

I remember having to do that Schneider shuffle in the early 80's when I needed a 150 enlarging lens. Tested 6 or 8 to find a good one. A couple were horrendously bad, a couple were very good, and the remainder fair to middling. My 80mm and 100mm lenses were Componons, but both were excellent ones on the first try. For each of those I chose the best of three, but in both cases there was not much variation in quality that I could detect.

Would have considered a Nikkor 150, but really wanted that nifty stop-down lever that only the Componons had.

We are probably close to the limits on what we can expect regarding mechanical tolerances in the SLR world for the price we are willing to pay.

I have had issues with a large proportion of "consumer level" SLR gear and far less with professional grade stuff, although there is some variation there too.

The other answer of course is to design systems that are self-calibrating. Without a doubt the biggest advantage of mirrorless cameras is that the AF is taken directly off the sensor. Focus accuracy on my Xpro1 is considerably better than I could achieve on a D7000, especially at wide apertures.

Now all we need is a new way to build flexible lenses whose focal length, shape and RI can be controlled electronically...

Of course electronics has tolerances, a design life and a reject rate too, but to a certain extent correction can be built in to the system (eg. pixel mapping) and testing can be automated to a greater extent.

Reminds me that a London UK camera store (i think it was RG Lewis) in the early 70s use to advertise that they tested before selling, all the lenses they stocked to guarantee you did not buy a 'lemon'. It was their way of being able to sell cameras/lenses at a higher price than the emerging camera discount stores at that time.

Dear James,

This was a point I made, also, in my columns talking about why camera bodies mattered and were more than just boxes that held lenses. One of the differences that mattered to pros was how well a camera body held its settings in use ( shutter speeds, focus accuracy, lens alignment ). All things that would get shifted slightly when a camera was bounced about.

You could find amateur models that would perform just as well when tuned up (e.g., my Pentax ME Super), but they wouldn't stay that way very long. Whenever I needed to do a lens or film test, I always retuned my ME Super, even if it had only been a week since the last test. I always needed a bit of a new tweak to bring it back to "perfection."

pax / Ctein

Dear Davide,

Yup. I hope I remembered my statistics right. (If I didn't I *know* someone will let me know.)

pax / Ctein

FYI, when I was a wee lad in the studio business in the early 70's, the Schneiders were truly falling off the cart as far as lens quality was concerned...we used to test all lenses from our local supplier before we bought any, and I can tell you we went through more than a few of each focal length before we picked one to keep...I tell people all the time, the success of Nikon as a view camera lens company had nothing to do with price and a whole lot to do with quality, or maybe just an average higher quality per batch...I've seen some truly appallingly unsharp Schneiders in my lifetime, and a lot of studio people I know wouldn't have touched the Rodenstocks with a ten foot pole at all for any reason...

BTW, I've rarely found, or used a "panda" Schneider (silver front, black back) from the 60's, that wasn't just fine...go figger...

Now that my eyes are temporarily sharpened to this topic I noticed that that lens rental business fellow has just "tested" his first batch of Canon 40mm f2.8 pancakes. It looks good in performance and consistency!

Mmmm...I like consistent batches of pancakes.....mmmm!

The comments to this entry are closed.