How Much Does a 5-Star Review Cost? Not Much.

Word-of-mouth is about the best advertising there is -- I trust geeky friends of mine to give me advice on new toys to purchase (or not), and they trust me with car-guy questions. But just how far beyond your real circle of friends can you trust "word-of-mouth" reviews?

I've spent many years in marketing research, and the most important thing I learned, and for you to remember, is: "Lies, damned lies, and statistics". I used to tell clients -- who would inevitably try to shape the research for their own benefit -- that they should just tell me what results they wanted, and I'd get them.

"Oh, no," they'd protest, "We want to know what our customers really think."

Really? I didn't think so then, and I don't think so now. They generally want to know what will make them look good.

When I pick up my car after a regular oil-change, the service adviser reminds me that I'll be getting an online questionnaire in the next day or two, and that "We strive for 100% outstanding, so I hope that's what you'll be able to give us."

Hint. Hint.

(In fact, I never complete those questionnaires. Because of all my years in research, I know too much about how they're constructed, and how easy it is to game the system. A properly-constructed questionnaire should ask you, right up front, if you're in the marketing, sales, or research of that particular product or service, and if you say, Yes, should disqualify you. If they don't ask those questions, it's not well-designed research. Just saying....)

Now my service adviser has never offered me cash for a positive review, or a free car wash, or anything of the sort, thankfully.

But there are apparently plenty of folks out there buying positive reviews. I've written before about Amazon's army of reviewers, and the gifts some have received for those glowing reviews.

Today's New York Times has an article by David Streitfeld on the problem of buying five-star reviews.

According to the article, some customers who purchased Kindle Fire covers from VIP Deals (for less than $10, plus shipping; "regular price", $59.99) were refunded their purchase price for a five-star review.

While the letter [from VIP Deals making the refund offer] did not specifically demand a five-star review, it broadly hinted. "We strive to earn 100 percent perfect ‘FIVE-STAR’ scores from you!" it said.

As Streitfeld drily writes, at least one customer who was offered the deal had her doubts:

"I was like, 'Is this for real?'" she said. "But they credited my account. You think it’s unethical?"

Um, that would be Yes.

Connections between the person who makes a product or offers a service and the person who promotes that product or service must (by law, not just ethics) be disclosed. Streitfeld writes that the Federal Trade Commission has starting cracking down on some of the most egregious examples, and fears that the examples it found are "far from isolated instances."

A technological solution may be on the way: academic researchers are now "trying to devise mathematical models to systematically unmask the bogus endorsements."

I like the idea, although I'm just cynical enough to suspect that the moment such an algorithm is devised, someone else will have designed an algorithm to get around it.

Maybe we should rethink hive-mind reviews altogether. I have more confidence in the Times' restaurant reviews than in Yelp, and more confidence in what Consumer Reports says than in glowing reports on Amazon. Because even if I don't know those reviewers personally, I know what they stand for.