The Fault in Amazon’s Stars
Lessons learned from a one-star rating
After much blood, sweat, toil, and tears, I self-published my first children’s book. Once the initial wonder, awe, achievement, and pride had worn off, I did what any self-respecting author would do: I listed it on Amazon.
The next day, it had a one-star rating.
One star (*). Out of five possible stars (*****). That is very few stars.
Someone must have really hated it, which surprised me for two reasons: 1) I think it’s pretty much the best book ever written by a human person, and 2) Literally no one had bought it yet.
When I first saw the rating, I was a little perplexed. I mean, I know that teaching kids about supply chain management concepts through means of a plucky, purple-haired heroine and her canine companion isn’t everyone’s cup of tea, but I didn’t think it would spark full-on outrage.
If you don’t like it, cool — buy your kid one of five zillion books about caterpillars instead. But to leave a one-star rating preemptively, out of spite, on a book you’ve never read (and, let’s be honest, never even heard of, because my author platform is not quite where it should be1)? Who does that?
That’s what I tried to find out.
I didn’t succeed, of course, but I did learn some lessons along the way. And if you can learn something from my misfortune, then it will not have been for naught.
Ratings < Reviews
Amazon switched from reviews to ratings late last year. You can now rate a product — whether you’ve bought it or not — without leaving any textual feedback.
Ostensibly, this encourages more shoppers to leave feedback because it removes the hurdle of having to formulate a coherent thought to make your opinion known. And, as we all know, more = better, always, in every circumstance, without exception.
The problem, though, is that ratings without reviews provide very little usable information.
My one-star rating is not actionable. If it’s legitimate, which I doubt, because it appeared before anyone had actually bought the book, it still only tells me that someone hated something about the book. Without an actual review, I can’t fix it.
Did they not like the story? Were the illustrations too busy? Was my depiction of dragons as fire-breathing and rambunctious biased and one-dimensional? I knew I should have found a sensitivity reader.
I’m not afraid of criticism; in fact, I really like it. And yes, I know that’s one of those things you’re supposed to say, like answering “Perfectionism” and “Working too darn hard” when the interviewer asks about your weaknesses. But I actually mean it.
Feedback fuels improvement. Sure, it’s nicer when it comes sandwiched between some hearty, glowing praise, but you at least need the meat. You need to know what people find objectionable so that you can [decide whether or not to] change.
Granted, nobody owes you feedback, and everyone is entitled to hate things for no reason, but star ratings just aren’t very helpful. They aren’t helpful to sellers, because they don’t know what to improve, and they aren’t helpful to buyers, because they don’t know what went wrong. Reviews are infinitely better.
So why the switch?
Because star ratings make sense…for Amazon. Guess which is easier to deal with: free-form reviews from human people — full of sarcasm, typos, and passive-aggressiveness — or {1,2,3,4,5}?
Whereas you have to employ actual thinking data scientists to parse written reviews and create some sort of meaningful categorization of them, you can have a precocious twelve-year-old create a linear regression for star ratings in two hours for a $100 gift card and an assurance to his tiger mom that it will look good on his college applications.
Here’s what it says under my sad little bar chart:
Amazon calculates a product’s star ratings based on a machine learned model instead of a raw data average. The model takes into account factors including the age of a rating, whether the ratings are from verified purchasers, and factors that establish reviewer trustworthiness.
This machine-learned model might work for products with hundreds of ratings, but not for my little book. I have one rating, not from a verified purchaser, and the result is that my book has one star.
Mein Kampf has four-point-four.
According to Amazon’s model, I am literally worse than Hitler.
Amazon Is Not Your Ally
As mentioned, this lone star appeared before a single copy had been sold. Since no humans knew about the book, I assumed the rating came from a bot.
It is in Amazon’s best interest to have a well-functioning ratings system; trust is essential in any sort of commerce, and arguably more so in e-commerce. I want accurate ratings, and Amazon wants accurate ratings. Naive little me figured we could unite around a common goal.
So I called Amazon Customer Service.
This was a star rating only, not a review, and those are anonymous, I was told.
Well, yes, ratings are anonymous, but I assumed that Amazon would at least know who did it. And I was very clear that I wasn’t asking who it was — I just wanted them to investigate whether it came from a suspicious account.
But it’s anonymous, my customer service rep reiterated. Evidently, nowhere in its vast AWS cloud cover does Amazon keep simple, transactional, small data like who gave what how many stars.
We had reached the limits of what my customer service rep could do, so she counseled me to email Seller Central or Author Central. I did so, and received — within seconds — the following response.
Please be informed that Amazon strives to cultivate a diversity of opinion on our products while staying within our guidelines. We allow our customers to air their honest thoughts on items they have received.
I get it, I really do. Ratings systems would be pretty worthless if people could have the bad ones removed. But, as I pointed out in my reply, this rating could not reflect an honest thought because no customers had received my item yet.
Shouldn’t a company as mature as Amazon have some sort of mechanism for separating out legitimate feedback from illegitimate? Probably. But why would they? You’re still going to buy stuff from them anyway.
Cutthroat Competition
The fact that someone can leave an anonymous one-star rating on a book that has never been sold, and that an algorithm displays it, is infuriating. The fact that you can’t access a human being at Amazon who has the ability or authority to do anything about it is even more infuriating. But perhaps the most infuriating of all is the fact that someone would do something like this in the first place.
I’m 99% sure that this single star is illegitimate, but I have to consider the possibility that it’s real. Even though no one had bought my book, there were at least eight people — all family and friends — that had read it.
One of them could, I guess, harbor such vitriol that they lied to my face and told me they loved it, only to run to their computer at the first opportunity to try to sabotage my dreams.
But the more likely explanation is that one of the 5,549 authors ranked ahead of me in Children’s Jobs & Careers Reference Books sees me as a threat. It’s a cutthroat category — I knew that going in — but I had no idea it would be this ruthless.
This is actually a thing — competitors leaving poor reviews on your product so that theirs looks better by comparison. It’s so prevalent that Amazon’s Community Guidelines specifically call it out, and give you at least the semblance of recourse.
Unfortunately, these guidelines are set up to govern reviews, not ratings. Reviews can be disputed, because there is something to dispute. There are words and patterns and indicators that mark a review as suspicious. Malicious reviewers write enough rope to hang themselves with, at least sometimes (although it sounds like the Report Abuse process is not particularly robust, either).
Since ratings are anonymous, you can’t prove that they came from a competitor. There isn’t even a dispute process. Which means that I’m stuck with a horrible rating that’s not legitimate but won’t be removed.
One Star is Better Than Zero
Every time I look at my product page, my stomach hurts. I have a visceral reaction to the injustice; the one-star rating will always be there, bringing my average down. It impacts my ranking and visibility, and might very well affect someone’s decision to buy my book or not.
But Amazon doesn’t get to determine my self-worth.
I choose to look at my one-star rating as a badge of honor. The hardest leap is from zero to one — just ask Peter Thiel. I used to have zero stars, because I used to have zero books…but now I’m at least in the game.
My book is on Amazon2. It’s a real, tangible thing, born of my intellect and hard work. It may never be a bestseller, but it might make a difference to a couple of people — and that’s enough.
Writing takes grit. Publishing takes even more. Once your book is out there, there’s no telling how people — or bots — will react. Criticism, legitimate or not, is part of the game. Smooth sailing makes for subpar sailors, and if you’re writing books that everyone loves, you’re probably doing it wrong.
There will be disappointments, and there may even be grave injustices. But there will also be pride, and accomplishment, and the knowledge that your words have made an impact.
So don’t worry about the naysayers (unless they have actionable feedback, and then maybe at least consider it). Don’t worry about the Amazons and the unfairnesses and the hiccups in the process. Write for yourself, write for the handful of people that will get something out of your work, and write to have written.
Because you know what they call the person who graduates last in his class at medical school? “Doctor.”
And you know what they call the person whose book is number 2,947,837 on the Amazon Best Sellers Rank?
An author.
1 Shout out to my 11 Twitter followers! ↩
2 And also on Gumroad, because I’m not a masochist. ↩
Occasional emails from Megan
I promise not to spam you and I promise not to be boring.