Quantcast

Tag Archives: wine competitions

Wine competitions, judging, and blind luck

winetrends

Wine competitions, judging, and blind luckOr, as the co-author of a new study told me: “Consumers should disregard results from wine competitions, because it’s a matter of luck whether a wine gets a gold medal.”

That’s the conclusion of Robert Hodgson, a winemaker and statistician whose paper (written with SMU’s Jing Cao) is called “Criteria for Accrediting Expert Wine Judges” and appears in the current issue of The Journal of Wine Economics. It says that those of us who judge wine competitions, including some of the world’s best-known wine experts, are ordinary at best. And most of us aren’t ordinary.

Because:

… [M]any judges who fail the test have vast professional experience in the wine industry. This leads to us to question the basic premise that experts are able to provide consistent evaluations in wine competitions and, hence, that wine competitions do not provide reliable recommendations of wine quality.

The report is the culmination of research started at the California State Fair wine competition at the end of the last decade. The competition’s organizers wanted to see if judging was consistent; that is, did the same wine receive the same medal from the same judge if the judge tasted it more than once during the event? The initial results, which showed that there was little consistency, were confirmed in the current study.

More than confirmed, actually. Just two of the 37 judges who worked the competition in 2010, 2011, and 2012 met the study’s criteria to be an expert; that is, that they gave the same wine the same medal (within statistical variation) each time they tasted it. Even more amazing, 17 of the 37 were so inconsistent that their ratings were statistically meaningless. In other words, presented with Picasso’s Guernica, most of the judges would have given a masterpiece of 20th century art  three different medals if they saw it three different times.

“This is not a reflection on the judges as people, and I don’t mean that kind of criticism,” says Hodgson. “But the task assigned them as wine judges was beyond their capabilities.”

Which, given the nature of wine competitions, makes more sense than many doubters want to believe. Could the problem be with the system, and not the judges? Is it possible to be consistent when judges taste 100 wines day? Or when they taste flight after flight of something like zinfandel, which is notoriously difficult to judge under the best circumstances?

When I asked him this, Hodgson agreed, but added: “But we don’t see an alternative. But it is an inherent problem. You just want to see the competitions give the judges sufficient time to do it.”

Perhaps. But my experience, after a decade of judging regularly, is that the results seem better (allowing for this um-mathematical approach) when I judge fewer wines. That means that the competition is smaller, or that the organizers have hired more judges. Maybe that’s where the next line of study should go, determining if judging fewer wines leads to more consistent results.

Colorado Governor’s Cup 2014

hr_vineyard_mtns

Colorado Governor's Cup 2014Ten years ago, when I first tasted Colorado wine, I spent much of my time being polite. As in, “This is nice. Thank you for letting me taste it.”

Those days are long gone, as was amply demonstrated last weekend during judging for the fifth annual Colorado Governor’s Cup. The red wines were exceptionally strong, and though the whites weren’t as good, they were technically sound and professionally made. In the regional wine business, that’s an accomplishment.

The best reds were cabernet franc and petit verdot, two Bordeaux grapes that do well in Colorado and that the state’s winemakers have taken to with enthusiasm (and especially cab franc). My panel gave a gold and double gold to cab francs, and a gold to a petit verdot. And the best wine of the competition was a petit verdot, from Canyon Wind Cellars. The results are here.

The wines were varietally correct, but also distinctive and reflected Colorado’s terroir — not a lot of fruit, more dry than a California wine, yet complex and very long. This is not an easy style of wine to make, but the state’s winemakers have made great progress figuring out how to work with their terroir over the past decade.

Finally, a few words about my pal Doug Caskey, who oversees the Colorado Wine Board and has run the competition since it started. One reason I enjoy judging this event so much is that Doug brings together judges who understand that Colorado wine isn’t French wine or California wine and isn’t supposed to taste like it came from those places. Sadly, too many judges downgrade wines that are “different,” which has nothing to do with quality, but with a preconceived notion about what wine is supposed to taste like that borders on snobbery and elitism.

The two people on my panel, Tynan Szvetecz and Sarah Moore, were terrific in this respect, and it was a pleasure to judge with them. I’m always lucky to work with people who put up with my idiosyncrasies, and they were no exception.

Dallas Morning News TexSom Wine Competition 2014

Dallas Morning News TexSom Wine Competition 2014

Dallas Morning News TexSom Wine Competition 2014The best piece of advice I ever got about wine came shortly after I started doing this, and from two different people (both well known for their irascibility). “The minute you think you know everything about wine,” each told me, “go do something else. Because as soon as you think that, you’re not capable of being any good at wine.”

In other words, shut up and listen to people who know more than you do, something I’ve tried to do as often as possible for the past 20 years. I had an excellent opportunity to do it again on Monday and Tuesday when I judged the 30th annual Dallas Morning News TexSom Wine Competition. Regular visitors here know how I feel about high-alcohol, over-ripe, and over-oaked wines. So what did I get to judge? Lots of high alcohol, over-ripe, and overoaked wines — 41 zinfandels from Lodi, Dry Creek, and Napa in California among 185 wines over the two days . And you know what? I liked some of them, and even gave two gold medals.

More, after the jump:

Powered by WordPress | Designed by: suv | Thanks to toyota suv, infiniti suv and lexus suv