Payola

By Ctein

Got a package today from Other World Computing (www.macsales.com) with an external hard drive adapter for me to test. It'll likely figure prominently in (probably) my next column. Accompanying it was a letter suggesting I keep the adapter after I tested it. I looked at the invoice and it was only a $40 item, so I said to myself, "Cool!"

Different realms of the review business have different customs about this. The two I'm familiar with are (personal) computer stuff and photography stuff. The former went from one extreme to the other. In the very early days of computer magazines pretty much anything went. Partly that was because it was a very casual, cozy group and gigabucks were not yet involved; partly because probably less than 100 people were responsible for 90% of the important stuff that was happening. You couldn't swing a cat at a gathering of journalists without hitting three people who were involved in important startups.

Eventually the bucks got much bigger, and serious conflicts of interest surfaced. Some very, very bad reviews got published. The magazines realized they needed to get their act in order and so it became total hands off, no business connection of any sort between writers and the products they review, past or present, and definitely no freebies (beyond innumerable coffee mugs).

The $100 rule
For at least 30 years in the photography writing business, stuff under $100 you got to keep. Stuff over $100 you sent back. Partly moral, partly practical. Few people were likely to compromise their reputation, ethics, and future income for <$100. Few manufacturers wanted <$100 items back; they told us explicitly that for what it would cost them to restock they'd rather we threw them out. It's still a pretty good rule of thumb. It's also kept the business pretty honest (there are exceptional individuals, both saints and sinners).

Hardly any publications purchase the products we test. Contrary to some readers' fears, manufacturers do not try to game the system; we don't get cherry-picked goods. Believe me, with all the quality control problems I've seen over the years, I can assure you of that! Kodak one sent me a scanner with a cracked filter (in all fairness, it'd been to a lot of trade shows). Beseler once sent me an electronic enlarger whose power supply caught fire in my darkroom!

A standard perk is that most manufacturers will sell you a product that you tested at the dealer or internal company price if you want it. This does not incline anyone to write more favorable reviews (there's no incentive to buying a product you don't really like) but it's induced many of us to keep an eye out for products that we think we might like.

Online reviewing practices seem a little different. An unusual percentage of the reviewers are wealthy enough to be able to purchase any products they test (back in the days of just paper publications, the phrase "wealthy reviewer" was an oxymoron). Some of them won't even accept products directly from the manufacturer to be reviewed; they insist on buying on the street. Good for them. No harm done, but in my opinion not usually necessary either.

On occasion there are real slipups, though. We narrowly avoided one at Camera & Darkroom in the early '90s. I got an excited call from my editor:

Dan (not the real name): Ctein, how would you like a studio strobe setup?

Me: Sure, what's up?

Dan: StrobEx (also not a real name) just called me and offered to provide all our senior reviewers with studio strobe kits.

Me: I could definitely use one. What's it going to cost?

Dan: That's the great part; they're free. All they ask in return is that when we write articles and we include any studio shots, we mention that we used their strobe unit.

(Modest pause at my end while I think of a delicate way to respond, because Dan is really, innocently excited about this.)

Me: We can't accept them.

Dan: Why not?! (Slightly plaintive note in his voice)

Me: (with great gentleness) Dan, have you ever heard the word "payola?"

I could hear Dan's face falling across several hundred miles of phone line.

Of course, we didn't accept the strobe units.

Scruples are such a burden.

_______________________

Ctein

Comments

[Peter]

Unfortunately there's a objectivity problem when the reviewer is going out and buying something on their own with their own money.

Either you have someone who just paid good money for something and they wind up viewing the product through rose colored glasses, because no one wants to feel like they paid good money for a lemon. Or the exact opposite reaction: you have someone who paid good money for something who thought it was going to be their "magic bullet", and because they're disappointed they just rip it to shreds.

[mcananeya]

Peter,

While that may be a problem for amateurs writing product reviews on Internet fora, I'm less convinced it's a problem among professional reviewers who get to see and test multiple versions of the same product from various manufacturers. In those cases, I'm willing to bet that a professional reviewer has bought a product because AFTER testing the alternatives, s/he thinks it is the best product (or deal) and therefore worth spending their own cash on. In fact, I would rather buy something that a professional reviewer plunked down their own cash for, than a product that s/he handled for 20 minutes and thought was "a neat idea" and therefore worth recommending (possibly rushed due to a looming deadline).

Best,
Adam

[e]

What an excellent way to signal to all vendors that the invoices they attach to all test products must be $99 or less!

[Eamon Hickey]

Hey, Ctein, glad you mentioned two of my (minor) bugaboos in regards to popular notions about camera (and other technology product) reviewing. One is the "so, do you get to keep all that cool stuff?" question. Nope.

But more amusing to me is the persistent one about cherry-picked review units. It is, as you said, much closer to the opposite; people would be amazed at some of the beat-up junk that gets sent around from reviewer to reviewer (and from the biggest-name companies, too). I've never met a professional reviewer who wouldn't confirm that. In fact, from the public's perspective, they should assume the review unit probably wasn't as good as a typical new one on a dealer shelf and judge the review with that in mind.

But I will come clean on some things: there is some good grub to be had in the reviewing game (Epson, bless their little 8-color hearts, took me to Nobu one night, for example); there's the odd boat ride around Manhattan; the usually attractive PR reps always pretend to think your insights are interesting, your jokes are funny, and you are likable. Lastly, there is one nice racket in the reviewing game: software. Although Adobe says that you're supposed to destroy a review copy of their programs after you're done with it, nobody else that I know of imposes any restrictions. I have long been disappointed in myself for not taking better advantage of this opportunity for corruption.

Hmmm, gotta' remember to log in with my alias before I post this ...

[Hugh Crawford]

I worked as a small software startup once, and we sent out a review copy of our product, which cost tens of thousands of dollars and was so hard to install that we essentially installed it for the customer , along with giving them a license key to the database engine we used.

Eventually the review came out and it was very positive , comparing our product to several competitive products.

It was one of those checkbox feature by feature reviews. There was on feature that we had been showing at trade shows that I thought we had removed before it shipped that the reviewer liked. Nobody could figure out how it was that they actually liked that feature when it was so awful in testing. We also noticed that we scored well in ease of use and ease of installation with the note "N.A. - almost installs itself" which we thought was some sort of humorous reference to a tech installing it for them.

We then realized that not only could none of us remember installing the product, but there was no record of us ever generating a key for them. In other words the review was apparently based on the reviewer hanging out in our booth at a trade show.

When we would get complemented on the rave review , we would always say "That review was not based on the functionality of the currently shipping product which is much greater than the functionality the reviewer experienced."

[Gordon Lewis]

I used to do product reviews, mostly for photo and computer mags. Although none of my editors explicitly asked me to write a positive review of any particular product, I had little doubt that it wouldn't be wise to make a habit of slamming equipment submitted by major advertisers. On the other hand, products from major companies such as Canon, Nikon, Ilford, IBM, Sony, et cetra were more often praiseworthy than not, so overt bias was less of a problem than some might think.

A larger issue, at least in my opinion, is whether there is any such thing as an "objective" review, or whether objectivity alone is the holy grail of reviewing standards. Some reviewers and methodologies are more scientific than others, but the reasons one might prefer one camera or lens over another are often quite subjective--and why not? Most of us are buying this stuff to produce photographs, not accurate reproductions of test charts. We also have to use it, so what's light to you might be heavy to me; what I find sharp and snappy you may find harsh; what I find intuitive you may find incomprehensible, and so on.

In short, all reviewers have their biases and areas of expertise. The trick is to know what their biases are and whether they're in alignment with your own. For example, I'm more interested in what Mike Johnston has to say about normal to wide primes than tele-zooms, which he doesn't often use. Ctein is the guy I look to for reviews of imaging processes and software. If I ever want to know which medium format digital film back to buy, Michael Reichmann's "The Luminous Landscape" is the first place I'd look--and so on. I'm sure the rest of you folks can offer similar reference points (and some of you probably will).

[Ctein]

Dear Peter,

I apologize for my unclear writing. The reviewers I was talking about aren't necessarily reviewing products they bought for personal use; it's just their policy that if they choose to review a product, they go and buy it rather than asking for a sample from the manufacture. It's not a "personal" thing.

In the situation you're talking about a professional reviewer has no problem maintaining fairness. It's just something you learn to do if it doesn't come naturally. It's part of the skill set of being a good reviewer, and it's not particularly hard to learn.

It's also not entirely correct. One of the things a good reviewer is supposed to do is to be able to put themselves in the minds of their readers and that means in substantial part evaluating equipment the way their readers would feel about the equipment. And the readers, after all, to spend their own hard-earned money. So a good reviewer actually tries to imagine, "What if I had spent my cash for this?" The skill, of course, is to not let it overwhelm good judgment and fairness.

There is no such thing as an "objective" review. Don't even try looking for one. All you can hope for is fairness and factual accuracy and reasonable judgment. But every reviewer brings their personal interests and disinterests to the review. Unless the product is so trivial that every single feature and aspect of it can be covered in reasonable length, the reviewer will choose to focus on some aspects of the product and ignore others, based entirely on what they think is important. A good reviewer for you is one who cares about the same things you care about. A bad reviewer is one who doesn't.

For example, I care little about studio photography. I am also not fond of artificial lighting for photographs. Those two areas of disinterest mean that there are whole feature-subsets of products that I simply ignore. They're not important to me, and if they're important to one of my readers, well they just have to look elsewhere for that information because I'm not the person who should be writing about it.

But regardless, the "just paid good money" thing isn't a problem for a pro reviewer.

~ pax \ Ctein
[ please excuse any word salad. MacSpeech in training! ]
======================================
-- Ctein's online Gallery http://ctein.com
-- Digital restorations http://photo-repair.com
======================================

[Mike J.]

"What an excellent way to signal to all vendors that the invoices they attach to all test products must be $99 or less!"

...And that's very typical of reader cynicism, too, although I realize you might not have meant it seriously. Readers impute low motives to both magazines and advertisers far out of proportion to what's actually warranted, in my experience. Of course, all of us who worked for magazines could tell you stories. Lots of 'em.

Mike J.

[Janne]

e, I guess that might happen in marginal cases, but nobody's going to be fooled by a Leaf digital back loaner, or a Nikon D300 marked at $99.

I think the single most honest review method I've seen is the lens reviews at Photozone (http://www.photozone.de/). They (he) solicits lenses from his readers to be sent to him, tested, then returned. The beauty of it is, there is no conflict of interest in any direction (he doesn't have an emotional investment, and no company connection to the gear). The downside is mostly that the reviews may take some time.

[ctein]

Dear E.,

That actually happened once, I think! About 20 years ago, Beseler sent several of us samples of a new digital darkroom timer they were going to introduce. The invoice packed with the timer indicated that the list price was $99.95. Based on the selling price of other Beseler products and other companies' competing timers, we were dubious about this. We strongly suspect that the list price would be somewhere between $125 and $175. But we had no way to prove it, and Beseler had a well-deserved reputation for being an honest and ethical company, and none of us could really come up with a polite and appropriate way to call them up and accuse them of being cheats.

So, probably a minor gaming of the system.

It was a very nice timer; I'm still using it!

~ pax \ Ctein
[ please excuse any word salad. MacSpeech in training! ]
======================================
-- Ctein's online Gallery http://ctein.com
-- Digital restorations http://photo-repair.com
======================================

[Ctein]

Dear Eamon,

Oh yes, there are lots of perks and freebies in the reviewing game. That's part of the fun! Though I never did manage to get a ride on the Fuji blimp, even though I was good friends with the PR person in charge of scheduling that program. (And, by the way, does anyone know what happened to Carol Smith after she left Fuji, about five years ago? I'd love to get in touch with her.)

Consumables, of course, have no hundred dollar limit or return aspect to them. Manufacturers sent me bricks of film and boxes of paper for review all the time. Frankly, they send me more of that stuff that I can use. Every four or five years, I pull the surplus out of my deep freeze, box it up, and send it off to a high school I know that still has a darkroom.

Software is an interesting question. I treat it the same as a consumable, because its return value is negligible and the out-of-pocket cost to the manufacturer is small. I suppose the possibility for abuse exists, but I think any reviewer who went into the software-reselling business would quickly find themselves cut off.

Adobe is one of the more tight-fisted ones to deal with, because everyone and their cousin asks for free copies of Photoshop, but I've never heard of this business of destroying the software after I test it. Not that I would, but it's never even been mentioned to me. I do have to justify my software requests of them in a way that I don't with most other vendors, but since I don't ever ask for stuff that I don't plan to write about it doesn't present me with a problem.

I did encounter one software publisher who wouldn't provide me with a copy of a Photoshop plug-in I was interested in writing about, because they said they didn't bother with online reviewers. Since this was downloadable software and all they had to do was provide me an activation key, I think it cost them more time and effort to write me an e-mail declining my request than to simply provide me with the key. Their loss.

My practice is to not review software that I can't get for free, for the simple reason that I try to make a living off of my writing. If I had to buy the products I test, as a matter of course, the income flow would be negative.

~ pax \ Ctein
[ please excuse any word salad. MacSpeech in training! ]
======================================
-- Ctein's online Gallery http://ctein.com
-- Digital restorations http://photo-repair.com
======================================

[Mike J.]

"It was a very nice timer; I'm still using it!"

Ctein,
I still have mine too.

Mike J.

P.S. Beseler is a beautiful company run by wonderful people and everything they do is perfect.

[Chris Crawford]

Beseler was a great company. I still have my old 23cIIXL enlarger with the dual-dichro color head, which I bought new when I was in college 13 yrs ago. I don't use it anymore, skin allergies forced me to go to scanning my film, but I miss my darkroom. Is Beseler still around, or have they been killed by digital?

[the editor]

I am hoping that your hard drive adapter research might cast some light on the best options for scratch drives for Mac users.
Your recent scratch drive item about how to speed up PS was fascinating, but one wonders what's an iMac user, for example, to do?
Howard French

[Robert Edelman]

To quote Groucho Marx: "I have my principles. If you don't like them, I have others."

[Ctein]

Dear Howard,

Check out the complete article in the current PHOTO Techniques.

pax / Ctein

[Cateto/Jose]

Very nice article, but it misses a significant issue which is, in my opinion, the current form of payola: big bucks disguised under the form of publicity in magazines or websites. Nobody speaks openly about that, but everybody in the business understand it.

I could mention (is it really needed?) a recent example of a certain, very important and influent French photography magazine, which published a rather harsh review on certain focusing issues of the flagship model of one of the top dSLR companies. That very company suddenly cancelled all their ads to be published in that magazine. Nobody thought it was a coincidence. Therefore there was a cause/effect relationship (at least, to the eyes of that company).

I could also mention some very prominent photography review websites, which use to make very quick review of the top dSLR models from the top companies, while other models (similar or better in overall quality) from smaller companies often have to wait months and months to see their products reviewed. Is it by coincidence that such delays affect companies whose budget for advertising is lower?

I think we should not be fooled: sure, reviewers don't get cameras costing many hundreds/thousands of dollars/euros for free. But that does not mean that payola does not exist in 2008: it exists for sure, unfortunately, and it's disguised under the all innocent publicity campaigns.

[MasonR]

Good point about the condition of review units...if you're at the top of the reviewer food chain (Pop, Shutterbug, OP, etc.) you probably get the first units to arrive in the States before anyone else...often including provisos that the firmware isn't quite ready, there are some pre-release bugs, just ignore 'em, it'll all work fine in the release version. Trust us.

On the other hand, if you are a lower-priority reviewer, you will likely end up with a camera, lens etc. that has been in the paws of a half-dozen reviewers before you. I know one reviewer who brought a high-end DSLR into a rain forest to test it out, and dropped it. I pity the next guy who got that unit! At the very least, let's just say you should budget substantial time for cleaning off the dust, fingerprints and other surface schmutz before you do the product shots.

So much for cherry-picking!

[Mike J.]

Mason,
Most often I get sent brand new units to test...even here at TOP. The K20D and 35 DA Macro Limited I was sent by Pentax were both brand new. The only recent exception I can think of was a printer. I was told it was new but it was a repack. They quickly sent another, truly new the second time.

Mike J.

[Shahar Dumai]

Just as Cateto/Jose mentioned, I am also familiar with magazines (cars and motorcycles, in this case) that have lost ad space because of negative reviews, or that were otherwise manipulated by various vendors (upon which the publication relies for reviews, a major part of its content).

I think niche journalism has its own inherent contradictions. You can't write about photography without taking pictures, but in order to take pictures - you must use a camera of some brand... readers would like the publication to pay for itself by ads, etc. etc.

Personally, I wouldn't mind if MJ would receive his personal Olympus, Pentax or D300 for free. The personal integrity of the journalist is the counting figure for me.

As a side note - what do you make of the "ranking system" of some of the most prominent photography website (e.g. dpreview, which I highly esteem.)

When was the last time you've seen a product ranked under "8"?

Is it because there's "no point in making the makers upset"?

Something to think about.

[Jan]

Old fashioned virtues Ctein, I like that, if what you say is true that is, and i have no reason to doubt that one minute.

I do know bloggers, Pod-casters etc. whom all basically state their "good" intentions wit respect to reviews of products .... whilst noting this o their podcasts it almost makes you cry listening to their virtuous little souls..... until you come face to face with them, or rather their agents. Advertising/PR costs even on this level, you want your product mentioned or reviewed you've got to pay/invest/donate, now I am sure there are "honest" guys out there, thing is I've never met them.... nor do I think I ever will

[Charlie H]

On the issue of publicity as payola, I love how all the car mags who do the year end best of in each category of automobile limit the competition to new cars or trucks which came out in that calendar year. No matter that last years Chevy/Ford far outperforms this years Chevy/Ford, the clunker still rates as the car of the year. Just introduce cars in alternate years and everybody is happy. ch

[Ctein]

Dear Cateto & Shahar,

As I made clear in my column, I was talking ONLY about computer and photography magazines. I don't know what the practices are in other subgenres (cooking, cars, audio, gardening), but I do know they're different. What happens in those realms has as little to do with photo mags as the laws in the US have to do with the laws in France.

And...

What I didn't make clear (and should have) is that my knowledge only extends to US publications. What happens with pubs in other countries is also likely to be very different. Different laws, different customs, etc.

Personally, I don't think much of ranking systems for complex products. Looking at them impartially, there are damned few bad (meaning lousy value for their intended market and their price) cameras out there. Plus, most reviewers try to avoid reviewing obvious dogs, because it's just not fun (human nature-- comes with the territory).

All told, it's really unusual to see a product that doesn't deserve a 7 out of 10. And equally unusual to see one that deserves 10. It's not a very useful nor discriminating spread.

(Mind you, for *my* particular needs, the scale easily goes all the way from 10 down to negative numbers. Probably true for you guys, too. But a good review tries to encompass more of the range of readers' preferences and what they think is important than one person's likes.)

pax / Ctein

[Bill Pierce]

Ctein -

When I first started writing for the photo mags, I noticed that some of the folks who had been around for awhile had a lot of cameras. I suspected these were cameras that they tested and that for whatever reason were not returned. These were my suspicions because someone might have a Pentax with a 50mm, a Nikon with a 50mm, an Exacta with a 50mm, a Canon with a 50mm, a Contax with a 50mm, a Pentacon with a 50mm (actually a 55mm), e.t.c.. What kept me on the straight and narrow wasn't integrity but the desire to have one camera and a lot of different focal length lenses.

[Ctein]

Dear Cateto & Jan,

The squeaky wheel gets the grease. Writers and editors don't have the time nor money to try to ferret out all the cool products that are out there. They depend on companies to tell them. Companies that do a better job of keeping me informed of their nifty new offerings get more reviews for me, simply because I know about their cool products. I don't know about the equally-cool product from some little startup which doesn't have the resources to keep me and the other reviewers informed of their doings.

This is less than ideal, but I don't see any way to fix it. It doesn't indicate favoritism towards the large companies, just an unfortunate fact of life. I think there are at least as many reviewers like me, who prefer to give publicity to the small guys, as ones who favor the big guys. Probably even more; it's nice to be appreciated and know that you're doing someone some good, and the small operations are a lot more appreciative and benefit a lot more from our reviews.

You're correct that there is a lot of schmoozing that goes on between the PR people and the reviewers. And I will definitely give more press to a company that I think has nice people working for it than one that doesn't. Why not?! I can't review everything out there; I might as well go out of my way to reward the white hats.

But all good PR people understand there is a line you just don't cross. It's okay for a PR person to say, "I really hope we'll get some good coverage from you." It is definitely not okay for them to say, "I expect to get good coverage from you." The white hats understand this.

Not everybody, though, wears a white hat. But that'll be the subject of my next column (yeah, I've decided to delay the OWC product review one cycle). Yes, there will be dishing of dirt and naming of names!

~ pax \ Ctein
[ please excuse any word salad. MacSpeech in training! ]
======================================
-- Ctein's online Gallery http://ctein.com
-- Digital restorations http://photo-repair.com
======================================

[Ctein]

Dear Mason,

I'm not sure I've ever gotten anything but new (or seemingly so) equipment to test. The only exception that comes to mind was the aforementioned Kodak scanner, which they told me was a trade show demo unit.

This makes me wonder what the lifespan of a "review unit" really is. I know I treat equipment hard. I know a lot of other reviewers who treat equipment badly. I know of at least a couple of cases where I got the "review unit" right after someone who I know beats the hell out of their stuff and the unit looked like it'd never been touched.

I have this sneaking feeling that the companies tell us to take good care of the review equipment and do hope that we will, but what they really do is figure that they're going to have to write off most of the units they send out for review. I wouldn't expect them to tell us that; it would make our behavior even worse! [ destructive smile ]

On the other hand, when I went to Baja to photograph the solar eclipse in 1991, Pentax USA kindly lent me their Pentax 67 400mm EDIF lens. They took it around to trade shows to show off; fortunately the eclipse fell conveniently between two shows. They were extremely clear (politely so) that they needed that lens back promptly after I returned and they needed it back in exactly the same condition they loaned it to me. It was the only one they had in the US.

I have never been so terrified around a product in my life. Not only did I treat that lens like a baby, it never left my side. When I went into restaurants, I carried the case with me. I was not sure exactly what would happen to me if anything happened to that lens, but I was pretty well convinced it would be a Very Bad Thing.

~ pax \ Ctein
[ please excuse any word salad. MacSpeech in training! ]
======================================
-- Ctein's online Gallery http://ctein.com
-- Digital restorations http://photo-repair.com
======================================

[MasonR]

Hi Mike & Ctein,

That's good to hear, in both cases. But I already know you're both at the top of the editorial food chain! ;-)

Ctein, my guess about review equipment life span is that each unit is checked (hopefully) after each review, and as long as it is still in good shape it's sent to the next reviewer and so forth until it cannot meet whatever standard the manufacturer sets.

I also know about being asked to treat a special piece of equipment with care. It can be nerve-wracking when you bring something worth thousands of bucks into the field. I was quite nervous when I had a Leica M8 on loan to do street photography in NY. I wasn't so concerned about someone harming me if I'd been "busted" for taking their picture (it hasn't happened yet, but you never know) but rather that someone might damage the camera!

Suffice to say, I was more cautious than usual and didn't get many good street shots that day--but the camera was unharmed!

OK, enough fun...I'm gonna get some coffee now and put it in my Minolta Maxxum coffee mug and get back to work!

[arion]

Interesting that the $100 limit hasn't changed since the 1970s. $100 in 1971 is about $400 in today's dollars, and $100 in 1976 is about $300 in today's currency, so is it fair game to keep any product less than $300 now?

There has been massive inflation since then, but it's kind of sad/funny that the limit hasn't ever shifted.

[Ctein]

Dear Arion,

Hey, that thought did occur to me, and you've even underestimated the amount of real inflation since then. To maintain average parity, the limit would have to be between $400 and $600 today.

But those numbers haven't held true for photography, and especially not digital-based photography. The average price of a good camera plummeted between 1971 and 1991. Even without adjustment in the threshold, the price of an excellent pocketable camera like the Olympus Stylus Infinity/Epic came perilously close to the hundred dollar limit in the 1990s.

Digital has been even more extreme. Circa $500 will get you a decent enough low end dSLR or a very excellent compact/all-in-one eSLR. Or a usable desktop, even laptop computer. Or a couple of terabyte external hard drives. Or a better-than-low-end LCD monitor. Or a Super-B sized photo-quality printer. Or…

...Well, you get the point. An inflation-corrected limit allows for serious bribery and corruption. So, there is the argument for sticking with the hundred dollar limit. The idea is not to find out what you can get away with while staying within the "rules;" the idea is to ensure that you aren't tempted to get away with anything.

Understand, of course, that there is no overriding body that arbitrates this and sets a standard, not even really an informal consensus. It was just a number that seemed acceptable to folks, and I still think it's an appropriate number.

~ pax \ Ctein
[ please excuse any word salad. MacSpeech in training! ]
======================================
-- Ctein's online Gallery http://ctein.com
-- Digital restorations http://photo-repair.com
======================================