The online ratings you’re looking at to pick a restaurant or hotel? Useless. At least in the aggregate, that is. Looking at the average rating for a business or product on a crowd-sourced review site is the fast track to a poorly-informed decision. I’ll show you how to do it better.
But first, why are the ratings garbage?
I read a lot, and started using GoodReads a few years ago. It’s been a handy way to remember books I really loved and ones that were less amazing (although I haven’t rated many books one star, because when I hate something that much, I stop reading it).
Most books in the GoodReads catalog have average ratings somewhere around 3.5, give or take (exceptions include the Harry Potter series, which gets averages in the high 4s, and classics like Les Miserables). There’s a very narrow range of numbers actually appearing for average ratings. Where’s the cutoff between a book so good I want to read it and one so bad I need to avoid it? If you want to choose a good book, the average GoodReads rating is a fairly useless way to do it.
The same issue arises on other crowd-based rating sites like Yelp (where 4 stars is the most-frequently awarded rating and the average rating is 3.8 stars out of 5) and TripAdvisor (with an average rating of just below 4 out of 5). Unless a business is truly spectacular or terrible, people’s opinions on its quality will vary. My three-star resort might be your five-star vacation; after all, we’re different people with different tastes. Over a large number of ratings, this results in a very tightly clustered set of average ratings for all but the most extremely good or bad businesses, rendering the numerical reviews useless for making a decision.
The good news is, crowd-sourced reviews are still a great way to increase the odds that you’ll have a great meal. The tricks are:
- Don’t focus on the average review, unless it is extremely high or low. A business that has a solid 5/5 star rating across 100 reviewers is likely doing something special. A business with a 3.8/5 is more likely appealing to one segment of the market much more than another.
- Look for themes in the actual reviews. Many crowd-sourced review sites help with this by putting small synopses with bold-faced terms at the top of the review page. If a restaurant’s positive reviews consistently rave about seafood dishes, you may want to base your decision on that rather than an overall quality metric.
- Identify people who frequently review on the site and have similar tastes to yours. Read the reviews of five of your favorite places and see who else loved them. Now look at other places these folks reviewed. You can do the same with your least favorite places; who disliked them as much as you do? Take advantage of having something in common with specific reviewers.
- Identify subject matter experts. On GoodReads, use the Groups feature to find mystery novel aficionados and follow their reviews. On Yelp, can you locate someone who seems to really understand Asian cuisine and look to their reviews when you’re in the mood for pho? With TripAdvisor, the Forums might help you locate someone who travels extensively in a particular part of the world or shares your hobby of snorkeling.
I would love to see review sites do away with the omnibus numerical rating altogether. Not only does it not produce a helpful decision-making tool, but it’s also difficult on the reviewer. What happens if a restaurant has amazing food but slow service? Mediocre cuisine but one of the most attentive, friendly waitstaffs in town? I’d love to see more nuanced ratings systems that I would hope would also show more variation in average ratings. Until then, I’ll be paying more attention to review content than rating.
By the way, since originally drafting this post, I abandoned a novel about halfway (160 pages or so) through after reading a Goodreads review from a like-minded reader. The plot was spinning in a direction I found offensive, so I checked the reviews, and sure enough, I was in for 160 more pages of crap. No thanks!
What are your tricks for making the most out of user-generated reviews?