Wine

The truth behind Wine Spectator’s “significant efforts to verify the facts”

In the course of damage control, Wine Spectator Executive Editor Thomas Matthews and others have made several questionable statements. Although it had been my policy not to respond to them (or to the name-calling–“mugger” has been my favorite), there seems to be enough uninformed debate going on in the blogosphere that I wanted to set some of these facts straight here.

1. WS writes: “We make significant efforts to verify the facts…We called the restaurant multiple times; each time, we reached an answering machine and a message from a person purporting to be from the restaurant claiming that it was closed at the moment.”

If it’s true that WS called the restaurant’s number (+39 02 4074 6174) multiple times during their “efforts to verify the facts,” and had trouble getting through, why didn’t they ever leave a message? Or send an email?

The only message WS ever left at the restaurant’s number was left after the award had already been granted, by an advertising salesperson asking if I wanted to buy an ad (starting at $3,090 and going up to $8,860). I’ve posted that message as an MP3.

The ad department also emailed me–again, after I’d already won an award (but, interestingly, before the level of the award was determined). The email read as follows:

 

  

“Dear Restaurant Award Winner: Congratulations, your restaurant has been selected as a Wine Spectator 2008 Restaurant Award winner! All award winners will be listed in our Annual Dining Guide, published in the August issue of Wine Spectator, and will be mentioned in our restaurant database and online at www.winespectator.com. I am writing to ask if you have an interest in publicizing your award by placing an ad in the upcoming Restaurant Award issue. We will be creating a special advertising section where ads will receive premium positions located in the Restaurant Dining Guide. Space closes on Friday, June 6th. Special rates start at only $3,090 for a 1/8 page 4-color. The rate card and additional information can be viewed here: http://www.winespectator.com/2008RestaurantAwardRateCard. If you are interested, it’s a good idea to contact us as soon as you can to get the best possible position adjacent to your state. If you have any questions feel free to contact me.”

 

 

 

In short, the only “significant efforts to contact” the restaurant were made by Wine Spectator’s ad sales department, and were made after the Osteria had already won an award.

2. WS writes: “A total of 102 [out of 256] wines earned ratings of 80 points (good) or better.”

Setting aside the fact that the “reserve list” contained the most expensive and lowest-rated wines, this is their justification for why the list was “excellent” overall? If you look at all wines in Wine Spectator’s database, 167,710 of 190,071 wines ever rated–88%–earned 80 points or higher. Only 40% of Osteria L’Intrepido’s wines did. Even if you look only at the 117 wines on the Osteria’s list that were rated by WS, 102 of 117 (87%) earned ratings of 80 points or higher–lower than the average of all wines that WS has ever rated (88%). Is that their definition of “excellence”? Below average?

3. WS writes: “The restaurant sent us a link to a Web site that listed its menu.”

It would not have been possible to apply for the Award of Excellence without filling out the restaurant’s web site, phone number, street address, menu, wine list, and so on. As I explained in my initial post, I created a basic online presence for the Osteria. This included posting some basic information to Google and making one or two posts to Chowhound in February (though none in August, as the magazine suggests). I wanted to see if Wine Spectator’s level of expert scrutiny in determining award winners went beyond the sort of basic online research that anyone could do in two or three minutes.

Yes, this experiment was mischievous. Deception was required to expose what I saw as a wrong against the readers and the public. But the deception was hardly elaborate. Everything I did took only a couple of hours.

Wine Spectator’s readers and the public have systematically been led to believe that the Awards of Excellence represented an expert stamp of approval for a restaurant with a wine program that has been judged “excellent” by the magazine’s wine authorities, rather than an advertising scheme. You can decide for yourself, based on the evidence set forth here and in my previous post, which it is.

16 Comments

  1. May I add to your very interesting post that went to a ‘famous’ US restaurants a few years ago that had received and continues to receive the ‘WS Restaurant Wine Award’ but didn’t actually have any of the ‘great’ wines in stock.

    It went like this: “Sir, we are out of this and this one. Oh, I’m so sorry, we are out of this one too. Sir, let me see, if we have this one in stock. Sorry, may I suggest another wine.”

    Only 2 (!) wines from the entire first page of their ‘famous’ wine list were available.

    My reponse was: “I’m terribly sorry, but I’m out out of here too.”

    I paid for my mineral water and left. I can’t comment on their food as I have yet to return.

  2. h chou

    May I add that your book is also false advertising. Instead of truly comparing expensive wines to inexpensive wines, you basically rate a bunch of relatively inexpensive wines. I don’t understand how that is different from what wine spectator did.

  3. lintrepido

    The Wine Trials is a guide to inexpensive wines. That is why it rates inexpensive wines.

  4. h chou

    I read the book. First of all, it uses statistical tricks to get their p value to an acceptable level. Notice how they kept saying that there was a difference between the most expensive and least expensive wines, but that it was in the middle range prices that one can see the most difference. This is playing with the statistics in order to get a result that they like.

    Also, Goldstein never gave the full results of his study in his book. He did not tell us which expensive wine the cheap ones were rated against. That is pretty useless then if he picked the worst tasting expensive wine (there is such a thing) just to show that some cheaper wines can beat ONE expensive wine. When you write a book supposedly based on how corrupt everyone else is, you better include all the information. While Wine Spectator may be beholden to advertising dollars, Goldstein is also playing with the truth to make his own money.

  5. H CHOU:

    Goldstein also fails to mention that not every participant tasted the same wines.

    Ooops….

  6. i don’t like the taste of wine. i really love to be addict with it but i can’t. hehe

  7. Hi. graet article.
    Cheers, Peter

  8. anonymous

    dude. you’re my hero.

  9. Parantar ha detto,
    14 Ottobre 2008 a 1:20 am

    i don’t like the taste of wine. i really love to be addict with it but i can’t. hehe

    YOU MUST LOVE WINE. THERE IS NO OTHER WAY 🙂

  10. Interesting information which I can use for my current research on phone history. Adam

  11. Keep doing what you’re doing!

  12. geoff daum

    Yep – I would agree with that.. Thanks for the line.

  13. I must have somehow missed it! Guess I should do some other research before coming to a conclusion.

  14. Thanks a lot man

  15. I get your point, if they cannot do things the correct way then why bother. It just makes the entire process senseless.

  16. Anthony Taylor

    Robin
    just discovering “The Wine Trials” (2008 edition). Very interesting.
    Page 17 you discuss the blind tasting issue. Parker comes here to Gigondas every year (end August) for his annual tasting and has been doing so for as far back as I can remember. Up until a few years ago (3 I think) he would always taste blind, i.e. the bottles all had elastic “socks” covering them. He now tastes all the wines non-blind. This really shocks me. Despite 35 years of immersion in wine I totally agree with your conclusion that it is IMPOSSIBLE to judge the quality of a wine 100% objectively if you know beforehand what the wine is.
    For Parker, who claims to respect the highest moral standards of impartiality, I was really shocked to witness his abandoning one the most elementary rules of impartiality : not knowing a wine’s identity before tasting.

Leave a Reply

Theme by Anders Norén