We’ve written before that summarizing opinion is complex, and that understanding the meaning behind product reviews can lend more insight than averaging the “star ratings.” I encountered a terrific example of needing this context tonight.

In a situation that any working parents reading may relate to, by mid-December each year I’ve abandoned the fantasy of gifts purchased months in advance and am racing to online shopping destinations with quick shipping. My first stop tonight was for a video game for my son, a game that I remembered was pretty popular and well received. When I checked out Spore on Amazon, I was surprised to see a 1.5 star rating with over 3,100 reviews. Did I have the wrong game? I IMed a friend for advice.

Turns out, the game itself is pretty cool. But the DRM (digital rights management) designed to prevent piracy is a ludicrous opt-in, leading gamers to review it negatively. Here’s a sample:

First of all, the game incorporates a draconian DRM system that requires you to activate over the internet, and limits you to a grand total of 3 activations. If you reach that limit, then you’ll have to call EA in order to add one extra activation. That’s not as simple as it sounds, since when you reach that point EA will assume that you, the paying customer, are a filthy pirating thief.

So, the bad reviews make sense: there’s a strong negative associated with the game but it’s not about the graphics or the gameplay, features that might matter to me (or the gamer). Getting to the why (without having to read an adequate sample of the thousands of reviews) was vital. Automating the process might have been even better,

Oh, and the punchline? Apparently EA managed to deter potential customers, annoy their existing base, and still end up with Spore as the most pirated game of 2008.

Tags: customer voice, research, uh oh, uncategorized by admin
Comments

We’ve often noticed that people tend toward extremes when assigning online product ratings. For Amazon products with an theoretically average (3 star) rating, more than 65% of all ratings lump into either the best or worst score – a ‘bimodal distribution’ in stats-speak. There are many potential reasons: a lack of clear criteria for different ratings, a desire to influence the displayed score, shameless promotion, spite, and so on. Whatever the reason, the end result is that many rating systems are essentially a thumbs-up, thumbs-down proposition and often give misleading information.

Fortunately, the lack of sophistication doesn’t carry over from quantitative ratings to text-based reviews. From what we’ve seen in the distribution of ratings, we might expect text-based reviews similarly to espouse straightforward points of view. Instead, most text-based product reviews actually contain ‘complex opinion’— writing with at least one observation contrary to the overall argument. (There are some notable exceptions to this trend.)

As an example, in this Amazon listing for a Casio camera, 61% of the 46 English-language reviews mention both a positive and negative aspect of their experience. Considering the average review is only about four lines long, the amount of even-handedness is surprising. Other merchants, such as Best Buy, include designated fields in their review systems for Pros and Cons to encourage more considered opinions.

Tags: research by edward
Comments