Rating Systems

How do you use the various rating systems you encounter in your life? Do you use the full resolution of each rating system, or have you unconsciously developed a mapping of your own classifications to the rating systems being provided?

Numerous products incorporate rating systems. Ideally, these systems allow users to develop profiles that then help recall past products and experiences, as well as discover and select new products and experiences in the future. The most common rating systems I've seen include:

• 5 stars
• 3 Stars
• Thumbs Up/Down
• 100 Point

Ratings for collections

In Apple's iLife suite, both iTunes and iPhoto incorporate the 5 star rating system, and underneath the hood, these ratings are stored on a 100-point scale.

I find it interesting how similarly and differently I use these rating systems in each of these products. In iTunes, I reserve the 5 star rating for songs I consider true classics, songs I believe will hold up over time and come to represent the best music I've ever heard. With a modest collection of 3806 items in my Music tab, only 34 items are rated 5 stars, less then 1%. 373 items are rated 4 stars, just shy of 10%, and 1547 items are rated 3 stars, which represents less than half my library. I have smart playlists defined for each of these ratings, and generally tend to play music with 3 or more stars, and I rarely mark music with less than three stars. In fact, of the 6 potential ratings available, I only really need 4: none, good, great, awesome.

In iPhoto, I use a similar 5, 4, and 3 star system. 5 star photos represent the absolute best moments of my family's life. Photo's with 4 stars and great, and photos with 3 stars are worth sharing with my family. I make little distinction between 1 and 2 stars, and only both marking photos with those low ratings as a way to indicate a photo worth preserving. As with iTunes, I have smart lists in iPhoto that find all these photos at various ratings. As with iTunes, I'm not really using all 6 rating positions (0-5), but only need: none, save, good, great, awesome.

Ratings for entertainment

Because these are collections I curate, it makes sense that these ratings to be only positive. But there are lots of places where I want to capture a negative rating because I am not collecting things but rather experiencing them.

Netflix offers a 6-position rating scale, with dislike, none, and 1-5 stars. I rarely mark something as dislike, though I do remember rating a bunch of things early on to give the recommendation system something to chew on. And I rarely mark something less than 2 stars, where 2 stars is meh, 3 stars is good, 4 stars is great, and 5 stars is something I would consider buying.

Hulu's approach to ratings mixes a negative-to-positive spectrum on top of a positive rating scale, where 1 star is "Hated it" and 5 stars is "Loved it". I personally don't consider this system to be a good design… it feels very first-grade-ish, where even the worst performance merits a "star" for effort. Unfortunately, there is no antithesis of the star symbol, and so it isn't easy to craft a rating scale with stars at the positive pole and something metaphorically connected at the negative pole.

So far, all of the rating systems I've described have suffered a bit from too much resolution, in that there are more choices available than I need to express an opinion or make a classification. I have to mentally transcode my own value onto the rating system available to me. For shared systems, this creates aggregate rating systems are are noisy and course, because even with a vocabulary of only 5 words, people may still not be saying the same things to each other.

To contrast this, consider one of the better (IMHO) approaches to rating experiences, the San Francisco Chronicle's movie reviews "little man". Except perhaps for the empty seat, each of the icons is very clear and understandable when seen alone, and makes complete sense when seen in sequence. Moreover, there is no ambiguity, and I believe most people can express their own responses to a movie using this scale with little-to-no adjustment on their part. The resolution of the rating system seems just right.

Ratings for wine

Now we turn to the rating systems for wine… Ug! What a mess! In the mid-20th century, wines were often rated on a 20-point scale, often credited to UC Davis. Then in the late 1970s' Robert Parker started publishing wine reviews using a 100-point scale. Talk about too much resolution! What's the difference between an 82 and an 83? Nothing really, but because Parker has become so powerful in the wine industry, the difference between an 89 and a 90 can make or break a wine maker. For good or bad, the rest of the industry has pretty much jumped on board.

There's no denying that the U.S. population can apply their grade-school experiences to understand the value of 90 points as representing a good wine. It's unfortunate that, unlike movies or food or music or just about any other experience, people rely far too much on wine critics' ratings and not nearly enough on their own experiences to judge and compare wines. And the sheer inanity of the 100-point resolution compounds this fact for the average wine drinker.

For Clinks, we concluded several things: first, that the 100-point scale isn't going to be replaced any time soon (certainly not by a few eager wine drinkers who decided to write a cool iPhone app); second, that the cure for the Parker monopoly is the emerging democratization of wine reviews and ratings, aka Wine 2.0; and third, that people need a way to express their experience of a wine that was easy, unambiguous, and direct. So, we came up with a rating picker that focuses on the near-universal metaphor of thumbs up and down, which maps back onto the 100-point scale.

In this way, the 100-point scale is collapsed into a manageable resolution. By no means will this quell the growing debate about the value and validity of the 100-point wine rating scale, but we think is does help average wine drinkers rank their experiences more easily, which then helps them remember the good wines next time they're at the store. It will be interested to see how people react to it.