The truth behind Wine Spectator’s “significant efforts to verify the facts”
In the course of damage control, Wine Spectator Executive Editor Thomas Matthews and others have made several questionable statements. Although it had been my policy not to respond to them (or to the name-calling–“mugger” has been my favorite), there seems to be enough uninformed debate going on in the blogosphere that I wanted to set some of these facts straight here.
1. WS writes: “We make significant efforts to verify the facts…We called the restaurant multiple times; each time, we reached an answering machine and a message from a person purporting to be from the restaurant claiming that it was closed at the moment.”
If it’s true that WS called the restaurant’s number (+39 02 4074 6174) multiple times during their “efforts to verify the facts,” and had trouble getting through, why didn’t they ever leave a message? Or send an email?
The only message WS ever left at the restaurant’s number was left after the award had already been granted, by an advertising salesperson asking if I wanted to buy an ad (starting at $3,090 and going up to $8,860). I’ve posted that message as an MP3.
The ad department also emailed me–again, after I’d already won an award (but, interestingly, before the level of the award was determined). The email read as follows:
In short, the only “significant efforts to contact” the restaurant were made by Wine Spectator’s ad sales department, and were made after the Osteria had already won an award.
2. WS writes: “A total of 102 [out of 256] wines earned ratings of 80 points (good) or better.”
Setting aside the fact that the “reserve list” contained the most expensive and lowest-rated wines, this is their justification for why the list was “excellent” overall? If you look at all wines in Wine Spectator’s database, 167,710 of 190,071 wines ever rated–88%–earned 80 points or higher. Only 40% of Osteria L’Intrepido’s wines did. Even if you look only at the 117 wines on the Osteria’s list that were rated by WS, 102 of 117 (87%) earned ratings of 80 points or higher–lower than the average of all wines that WS has ever rated (88%). Is that their definition of “excellence”? Below average?
3. WS writes: “The restaurant sent us a link to a Web site that listed its menu.”
It would not have been possible to apply for the Award of Excellence without filling out the restaurant’s web site, phone number, street address, menu, wine list, and so on. As I explained in my initial post, I created a basic online presence for the Osteria. This included posting some basic information to Google and making one or two posts to Chowhound in February (though none in August, as the magazine suggests). I wanted to see if Wine Spectator’s level of expert scrutiny in determining award winners went beyond the sort of basic online research that anyone could do in two or three minutes.
Yes, this experiment was mischievous. Deception was required to expose what I saw as a wrong against the readers and the public. But the deception was hardly elaborate. Everything I did took only a couple of hours.
Wine Spectator’s readers and the public have systematically been led to believe that the Awards of Excellence represented an expert stamp of approval for a restaurant with a wine program that has been judged “excellent” by the magazine’s wine authorities, rather than an advertising scheme. You can decide for yourself, based on the evidence set forth here and in my previous post, which it is.




September 1st, 2008 at 3:21 am
May I add to your very interesting post that went to a ‘famous’ US restaurants a few years ago that had received and continues to receive the ‘WS Restaurant Wine Award’ but didn’t actually have any of the ‘great’ wines in stock.
It went like this: “Sir, we are out of this and this one. Oh, I’m so sorry, we are out of this one too. Sir, let me see, if we have this one in stock. Sorry, may I suggest another wine.”
Only 2 (!) wines from the entire first page of their ‘famous’ wine list were available.
My reponse was: “I’m terribly sorry, but I’m out out of here too.”
I paid for my mineral water and left. I can’t comment on their food as I have yet to return.
September 1st, 2008 at 4:41 am
May I add that your book is also false advertising. Instead of truly comparing expensive wines to inexpensive wines, you basically rate a bunch of relatively inexpensive wines. I don’t understand how that is different from what wine spectator did.
September 2nd, 2008 at 2:32 pm
The Wine Trials is a guide to inexpensive wines. That is why it rates inexpensive wines.
September 3rd, 2008 at 8:05 pm
[...] Goldstein writes, the magazine still claims to have checked its facts sufficiently. Reporters called the Milan phone number Goldstein had left but every time got his answer phone [...]
September 12th, 2008 at 7:16 am
I read the book. First of all, it uses statistical tricks to get their p value to an acceptable level. Notice how they kept saying that there was a difference between the most expensive and least expensive wines, but that it was in the middle range prices that one can see the most difference. This is playing with the statistics in order to get a result that they like.
Also, Goldstein never gave the full results of his study in his book. He did not tell us which expensive wine the cheap ones were rated against. That is pretty useless then if he picked the worst tasting expensive wine (there is such a thing) just to show that some cheaper wines can beat ONE expensive wine. When you write a book supposedly based on how corrupt everyone else is, you better include all the information. While Wine Spectator may be beholden to advertising dollars, Goldstein is also playing with the truth to make his own money.
September 17th, 2008 at 8:26 pm
H CHOU:
Goldstein also fails to mention that not every participant tasted the same wines.
Ooops….
October 14th, 2008 at 1:20 am
i don’t like the taste of wine. i really love to be addict with it but i can’t. hehe
November 18th, 2008 at 1:23 pm
Hi. graet article.
Cheers, Peter
January 8th, 2009 at 1:48 pm
dude. you’re my hero.
February 21st, 2009 at 5:02 pm
Parantar ha detto,
14 Ottobre 2008 a 1:20 am
i don’t like the taste of wine. i really love to be addict with it but i can’t. hehe
YOU MUST LOVE WINE. THERE IS NO OTHER WAY
May 13th, 2009 at 5:53 pm
Interesting information which I can use for my current research on phone history. Adam
May 15th, 2009 at 8:18 pm
Keep doing what you’re doing!
May 17th, 2009 at 10:01 am
Yep – I would agree with that.. Thanks for the line.
July 3rd, 2009 at 7:50 am
I must have somehow missed it! Guess I should do some other research before coming to a conclusion.
December 3rd, 2009 at 11:17 am
Thanks a lot man