This from the abstract:

“In markets for experience goods, publications exist to help consumers decide which products to purchase. However, in most cases these publications accept advertising from the very firms whose products they review, raising the possibility that they bias product reviews to favor advertisers…Although the average Wine Spectator ratings earned by advertisers and non-advertisers are similar, I find that advertisers earn just less than one point higher Wine Spectator ratings than non-advertisers when I use Wine Advocate ratings to adjust for differences in quality.”

These are wine ratings, not the restaurant Awards of Excellence, which I’ve written about in the past; the applicants for those awards are advertisers by definition (having submitted a $250 fee to be considered).

Karl Storchmann has also posted an interesting blog entry about Reuter’s paper on the AAWE website.

Reuter later retreats to a statement that he “finds little consistent evidence of bias…at worst, the tests for biased ratings suggest that Wine Spectator rates wines from advertisers almost one point higher than wines from non-advertisers. However, selective retastings can explain at most half of this bias and then only within the set of U.S. wines rated by both Wine Spectator and Wine Advocate. Given Wine Spectator’s claim that it rates wines blind, the remaining difference in ratings may simply reflect consistent differences in how the two publications rate quality, which leads to predictable differences in advertising. This interpretation is consistent with the fact that tests for biased awards provide no additional evidence of bias. Therefore, despite the fact that Wine Spectator is dependent on advertising revenue, the long-run value of producing credible reviews appears to minimize bias.”

I think this conclusion is softer than it need be. Even if selective retastings explain only half of the one-point bias, that’s still pretty damning; it means that if you advertise in Wine Spectator, you might well get the benefit of a selective retasting that gets you, on average, an additional half-point. Translation: advertising influences ratings.

With respect to the other half-point, if there are indeed “consistent differences in how the two publications rate quality, which leads to predictable differences in advertising,” then you should try leafing through a copy of Wine Spectator and seeing if you’d trust critics who favor the types of wines that tend to advertise in the magazine. I think the roster of advertisers speaks for itself.

The more important issue, perhaps—especially if you’re a small wine producer—is how difficult it is to get magazines like Wine Spectator to even review your wines at all. And this is where, anecdotally, bias might play an even larger role. “Unsolicited samples,” states the Wine Spectator website, “may not be tasted.” Advertise in the magazine, and that problem seems to go away.

And then there’s the matter of the selection of a wine (Columbia Crest Cabernet Sauvignon Reserve) from a Wine Spectator advertiser (Chateau Ste. Michelle) as this year’s Wine Spectator wine of the year.

Although proving bias in every such case is a complicated, difficult point, the obvious conclusion of all such research is the simplest:

We should be skeptical of criticism whose publication is financially supported by the producers of the products being criticized.

Wine critics should not accept advertisements from wineries.

Period.

Ethics scandals are politico porn. They’re also fertile ground for undeserved scapegoating. But there’s one category in which, across the board, there’s not nearly enough public stoning going on: the world of information intermediaries. On the government side, that means regulatory agencies; in the private sector, it’s the critics, the expert witnesses in capitalism’s de facto justice system.

Information intermediaries, we’re to understand, are society’s check against puffery. They make careers of trustworthiness and accountability. In society’s service, they apply rigor to the claims of corporations and analyze their standards. For this hard work, they’re rewarded by the marketplace and by the United States—sometimes handsomely, sometimes not.

Two bits of recent news bring about two otherwise disparate intermediaries, both preeminent in their niches—Robert Parker’s Wine Advocate, the publication whose critical appraisals are one of the central determinants of a wine’s success or failure on the marketplace, and the Federal Aviation Administration, the agency whose critical appraisals are the primary safety check against America’s airlines—systematically abusing that authority.

Jay Miller: Disfrutando?

Parker’s is one of the few wine publications that don’t accept advertising, for which he deserves praise. And it’s certainly acceptable to take free samples of wine from producers—that’s often the only way to taste new releases before they’ve gone to market. But the recent transgressions of Jay Miller, Robert Parker’s right-hand man, are spectacular indeed. In another classic case of the traditional print media jumping on the bandwagon of a topic that had been exposed quite a bit earlier by an incisive blogger—in this case, Tyler Colman, who goes by “Dr. Vino”—Miller’s series of all-expenses-paid vacation/junkets, financed by wine producers, have finally been reported by the mainstream media in a recent Wall Street Journal article.

Some of rumors about Miller’s behavior in Argentina go quite a bit further in scandalousness than the mere acceptance of free hotels, food, and drinks. But what’s crystal clear, as reported in the Journal and, previously, by Colman—and admitted by Parker—is that the writers of Parker’s Wine Advocate accept lavish free meals from, and are flown around on weeks-long junkets by, the same wine producers whose wines they’re supposed to be critically reviewing—in Argentina and elsewhere.

Parker doesn’t just present himself as a wine writer, and he doesn’t just present Wine Advocate as a wine magazine. No, these are “consumer advocates”: in Parker’s new statement of ethical standards, which was published on eRobertParker.com after the scandal broke, he writes: “I…remain today…significantly influenced by the independent philosophy of consumer advocate Ralph Nader.” Mr. Miller’s only job is to independently criticize wines and assign numerical ratings to those wines in service to, and he is engaged in the systematic, ongoing practice of being taken on vacation by those wines’ producers.

Miller has apologized, and Parker has tried to explain himself, but Parker’s actions (or lack thereof) speak more loudly than his rambling statements on bulletin boards. If it’s even true that Parker didn’t know about Miller’s junkets beforehand (which is extremely unlikely), the fact that Parker didn’t fire Miller when he found out is a loud, clear statement that what he did really wasn’t that bad. In fact, it’s endorsed even in the new code of ethical standards. Incredibly, rather than rejecting Miller’s trips, that statement specifically renders them acceptable:

I…expect [the Wine Advocate critics], as I have done for 30+ years, not to solicit or accept free hotel accommodations or hospitality not directly related to their professional endeavors.

Not directly related to their professional endeavors? Of course a free tasting junket would be related to their professional endeavors. That’s exactly the problem!

Parker finishes his exhausting essay with a poetic flourish: “Wine is, in the final analysis, a beverage of pleasure, and intelligent wine criticism should be a blend of both hedonistic and analytical schools of thought—to the exclusion of neither.” At least the record shows that he’s got the hedonistic part down, anyway.

Meanwhile, the New York Times reports that over at the F.A.A., the airlines being rigorously scrutinized for adherence to strict safety standards—again, the organization’s primary responsibility—are referred to within the agency as “customers”:

In 2008, two F.A.A. inspectors assigned to Southwest Airlines testified before Congress that their managers had let Southwest fly its Boeing 737s without inspections for cracks that the safety agency required. Office managers referred to the airline as the regulatory agency’s “customer.” Top F.A.A. officials eventually conceded that the inspectors were right and the middle managers were wrong.

The Times’ investigative report tells the chilling story of an inspector named Christopher Monteleon, who voiced serious concerns about the incompetence and low safety standards of the pilots of Colgan Air a year before one of the airline’s Bombardier Dash 8-400 turboprop planes crashed in Buffalo, apparently due to pilot error. Apparently, Monteleon’s warnings were not just ignored but actually punished by his superiors:

Three times, he said, the pilots flew the airplane faster than the manufacturer’s specifications allowed, but they initially refused to report this and have the plane inspected for damage…[T]hey tried three approaches to the airport in Charleston, W. Va., and “botched” all of them, failing to get the plane at an appropriate altitude, on the right path and at the right speed for landing. “They got confused,” Mr. Monteleon said…But when he reported problems to his F.A.A. superiors, he was suspended from important portions of his job overseeing Colgan’s acquisition of the Dash 8 and given a desk job, he said…Colgan crews were flying fatigued, Mr. Monteleon said, and were not fully focused on the tasks in front of them, two factors apparently in play in the Buffalo crash. All 49 people on board the flight, which took off from Newark, were killed, along with one man on the ground. Mr. Monteleon said his supervisors were too “cozy” with Colgan, and eager to help it keep its schedule…In one memo retained by Mr. Monteleon, his manager indicates that he was reassigned because of his “conduct during a work-related duty” and because “the matter also required management to immediately respond to the operator’s scheduling needs.” The operator was Colgan.

In aviation, as in wine, our intermediaries have failed in their duty to their real customers: the readers, the consumers, the citizens. And unlike what happens when ethics scandals directly involve politicians, neither Robert Parker nor the F.A.A. is likely to be voted out of power anytime soon.