How to improve wine competitions

Wine competitions have received tremendous amounts of criticism, whether it’s from unreliable results, results that seem odd, and results that the experts don’t like. Or, as the co-author of a study of competition failings told me, “Consumers should disregard results from wine competitions, because it ?s a matter of luck whether a wine gets a gold medal.”

Can wine competitions fix these problems and become more reliable? This is especially relevant given the recession, when wineries reduced the number of competitions they entered. This has led to a shakeout in competitions, and those that don’t adapt to the new conditions, where wineries want more value for their entry fees, won’t make it. I can’t emphasize this enough: Wine competitions are at a crossroads, where their results are increasingly irrelevant to consumers and less important to ever more wineries. The millions of people who buy Cupcake Red Velvet probably don’t even know competitions exist.

Hence the need to make the results more statistically valid, and the Wine Curmudgeon’s five suggestions, based on more than a decade of judging, to do that — after the jump:

? Pay the judges. Many competitions reimburse expenses, but don’t pay judges. In which case, we’re donating our time to very profitable endeavors. The biggest West Coast events generate close to a million dollars in revenue from entry fees, sponsorships, and public tastings, and even smaller competitions can gross in the six figures. If we’re helping you make money, you should pay us. Which is why I’m now only going to judge competitions that pay judges. My time is worth more than helping someone else make money.

? Use more judges. One reason for the outcry about results is that judges do 100 to 200 wines a day, which is too many. How can you give a wine a fair assessment when it’s one of a 25-cabernet sauvignon flight, and there’s a 25-merlot flight after that? This is also about cost. Hotel rooms cost money, and more judges means higher expenses. But doesn’t it make sense we’d get better results judging only 50 wines a day?

? Match the judges with the wines they judge. Roses tend to do poorly in competitions, because many judges don’t think roses are real wines. So why do those people judge roses? Or, as a judge at a competition I did infamously told us, he thought nothing but cabernet sauvignon, pinot noir, and heavily-oaked chardonnay deserved gold medals. So why was he judging grenache?

? Diversify the judges. This is not just about judging panels made up of old white guys, but about judges who appreciate different styles, different grapes, and different approaches to winemaking. If all you know and like is the modern California style, how can you be fair to wines made with different grapes made in a different style? Because those wines are entered in in California competitions.

? Use analytics. If they can do it in baseball, we can do it in wine. Develop ways to see if the results make sense or are just luck, and fix them if they aren’t. This is one of the biggest failings in the competition business today, the idea that since judges are experts, we don’t have to do anything to improve the results.