March 2016
I spend a lot of time on Amazon.com, partly because it’s a great place to research the consumer-electronics market, and partly because it’s often the most convenient and affordable way to find and buy things. I also read the Customer Reviews posted on Amazon, because they’re sometimes the only source of information about a product other than the manufacturer’s website. But three recent events left me doubting the reliability of these reviews -- and wanting to do a little checking for myself.
A few weeks ago, a representative of a company that sells Bluetooth speakers told me that he’d send me their latest speaker for free -- if I’d write a review about it on one of my various media outlets and post a Customer Review on Amazon. It wasn’t the first time someone had offered to give me a product in exchange for a review, but it was the first time someone had insisted that I augment my published review with an Amazon review. It made me wonder if anyone else had taken this guy up on his offer.
(For the record, all of the professional reviewers I know have at least half a room full of products whose manufacturers won’t pay to have them shipped back, so offering them free products isn’t likely to sway their opinions.)
When I mentioned the exchange to Lauren Dragan, headphone editor for TheWirecutter.com, she shrugged and told me that this sort of thing is normal. She directed me to iLoveToReview.com, a website through which a product manufacturer can arrange to have positive reviews posted on Amazon. The product manufacturer pays the site, which in turn pays its reviewers to buy the product through Amazon, so the review will carry the credibility of a “Verified Purchase” designation.
I doubt any established company would bother with such a site. But it must be tempting for new makers of headphones and Bluetooth speakers -- and new companies seem to appear daily -- who fight long odds to get their no-name products professionally reviewed.
To Amazon’s credit, the company has been trying to clean up their reviews, but it seems to me that those whose ingenuity is inversely proportional to their ethics can probably get around any virtual walls the company erects.
A search for the least-expensive multi-driver earphones uncovered another series of questionable reviews. The ActionPie VJJB-V1S High Resolution Heavy Bass in-ear headphones cost just $26.99 on Amazon but feature a driver configuration I’d never seen before: two 6mm dynamic drivers in each earpiece. I can’t think of a good reason for any earphones to use two of the same dynamic driver, and 6mm is much smaller than the drivers used in most earphones. It’s like replacing your car’s big V8 engine with two smaller four-cylinders, just so you can say you have dual engines. The images on Amazon seemed to be CAD renderings, not actual photos, but I could see that the ActionPies’ drivers fire into a common enclosure that didn’t look as if much work had been put into its tuning. And, I have to admit, the name ActionPie is way too Chinglish for me to take very seriously.
There was just no way these things could be any good; in fact, they looked laughably bad. Yet reviews of the ActionPies ranked them at an average of 4.3 stars. Then, at the end of one five-star rave review, I read this: “I received a free sample of these earphones in exchange for my review.” Something had to be up.
But my curiosity got the best of me. I ordered the ActionPies, and two days later they showed up in a decent-looking box along with seven pairs of eartips (one pair foam), a drawstring carrying bag, and a zippered semi-hardshell case. Most earphones come with only three sets of eartips, and I’ve reviewed $99 earphones that don’t come with even a case, so this was a nice surprise.
What was an even bigger surprise was that the ActionPies fit and sounded great. I’ve heard almost all of the major-brand earphones priced under $50, and many of the minor brands, and the ActionPies were better than almost all of them. The sound was generally neutral, without the gross tonal aberrations (boomy bass, screeching treble) of most cheap earphones. A slight extra kick in the lower to mid-treble subtly emphasized voices and snare drums, but that was all I found to complain about, and the effect was subtle enough that I quickly got used to it. Curious, I ran a frequency-response sweep and found nothing out of the ordinary -- and I must confess that I was hoping to enjoy the visual spectacle of wild response swings produced by the ActionPies’ weird arrangement of side-by-side drivers.
All this taken into account, what have I learned about trusting Amazon reviews? Not much. Professional audio reviews aren’t perfect, but the reviews on Amazon are much more subject to manipulation by manufacturers . . . and by their competitors. Some reviewers on Amazon are true experts with ample reserves of maturity and wisdom, while some will give two stars if the product arrives a couple of days later than expected. Or, worse, they’ll give one star without even having tried the product, merely because they have a grudge against the brand or the product category.
Still, even though we have to read Amazon reviews carefully and dispassionately, with all the skepticism we should apply to political candidates, I’m glad they’re there. Amazon reviews are a long, long way from reliable, but for now, they’re often the only way we have to sort through the hundreds of new tech products that hit the market every single day.
. . . Brent Butterworth
brentb@soundstagenetwork.com