Review score review
See that above? That number in the scorecard? That's the last one of those you'll be seeing on Engadget. (The last one for a while, at least -- "never say never" and all that.) Review scores were added to the Engadget way of doing things back in July of 2010 and, since then, they've had an overwhelming effect on how our reviews are read and perceived by you, our dear readers. We write our reviews and do our best to have the text within them speak to each and every one of you, but as we've learned those numbers truly only help very few of you.
So, as you may have noticed, review scores on Engadget are dead. Join us as we explore why. %Gallery-133663%
Hardware
A review score is a number, a single digit (we never did cover something worthy of a pristine 10) that gives a final, conclusive rating of the overall quality of a given device. That number is how nice that device looks, how well it performs, how heavy it is, how healthy its battery is, how much fun you can have with it and, of course, how much it costs.
A review score takes all that information and more, the subjective and the innate and the substantially complex plus the objective and the concrete and the easily comparable, and tries to rank it against the entire galaxy of other devices that may or may not compete directly or indirectly with it. More than that, the system attempts to scale an already established number based on the relevance of the device it was assigned to today.
That's of course because a review score never changes -- it never expires, has no shelf life, but is still distinctly perishable. Products that were deserving of a 8 or 9 last year probably wouldn't receive the same score today. Those numbers are stale.
Performance
A review score attempts to take a couple-thousand words worth of exposition that deeply analyzes the many and myriad features of a given device or service and boil all that down to a single digit. It does an incredibly poor job of it.
Take the Samsung Chromebook Series 5 review, for example. That's been one of our most controversial scores of late (we gave it an 8/10) -- and for good reason. What you have is a device that is well built, nice looking, affordable, and offers exemplary battery life. It's also a device that, at its core, runs nothing more than a web browser and is only useful when it can find a solid connection.
Do you need more than a web browser? If so, what a horrible idea for a device! This thing is like a 2, or maybe a 3. Do you do everything in a browser anyway, and live an always-connected lifestyle? Well, then this thing is awesome! Give it an 8, maybe a 9, and go play some Angry Birds in Chrome and stop worrying about trivial things like file systems and operating systems, man.
Anyone who took the time to read the review, the whole review, to delve through to the conclusion, learned that for themselves. Anyone who just looked at the score and had already made up their mind whether a Chromebook is for them either felt fully vindicated or fundamentally wronged. Their ultimate feeling depended of course on which of the two groups described in the last paragraph they fell within.
It's this nature of review scores, an inability to add qualifications based on the mindset and opinions of the reader, that make them not long for this world -- at least not our world.
The replacement
Don't worry, we know. You're busy. We're busy too. (Really, really busy.) You don't always have the time to read the full review and we know you want a quick way to at least get the gist of the thing. So, we're introducing the Bottomline.
This is a 140 character conclusion that will deliver the essence, the finely distilled flavor of the review in an easily palatable dose. Will it deliver every nuance of the full review? Absolutely not. That's why we still write the full review. There will also be links over to Amazon on items that are listed over there, where you're free to dig in and peruse the specifications, peek alternatives to a given device and maybe read some feedback from folks who have already purchased one.
Wrap-up
Quite simply: you can't summarize every detail of a full product review with a number -- even if you add two or three or four numbers split into categories or take those numbers and add decimal points. Abstract them as stars, crystals, slices of delicious apple pie or some other funny little graphic and it's just a different spin on the same old game. That game does a disservice to the full review, to the device being reviewed, and most importantly to you, the reader. You deserve to be better informed and not to be lured into making an instant judgement based on a digit.
So, review scores are being reborn on Engadget. We're going to keep investigating ways to recognize top-tier devices, to highlight those gadgets that clearly step above and beyond the rest, but the Bottomline is here now and it's here to stay. Yes, you still can't capture everything about a device in 140 characters, but you can sure do a hell of a lot better than with one digit.
