Introducing review scorecards!

As you may have noticed, here at Engadget we've stepped down hard on the gas pedal when it comes to reviews. We've been putting a lot of time and resources into delivering more product reviews at a faster pace, so you guys can know just exactly what you're getting into when it comes to the gear that we cover. In support of those efforts, today we're happy to introduce our review scorecards. We wanted to find a way to deliver a snapshot of our reviews in a really simple, glanceable manner, and also make those snapshots available to sort and search through. So starting immediately, we'll be including scorecards in any new review that goes up, and those cards will appear on our new reviews sorting page. Right now we've got some simple sorting options in there, but we're working on more in-depth sorting options that will allow you to drill down on specifics, so you can find exactly the product you're looking for. We're also in the process of moving backwards through our older reviews and getting cards in place, so if you see a few holes here or there, don't be surprised.

Some guidelines you should consider for these cards: firstly, the cards are based on our review at the time the post was published, meaning they're not relative to one another (unless the products were reviewed in the same category at the same time). That means that the G1 can have a higher rating than the Droid X, even though right now it's obvious that the Droid X is a superior phone. The Droid X is being reviewed now, and the G1 was reviewed then. Also, we want to provide a scale of what these numbers mean, so that there isn't any confusion. A 0 rating you'll likely never see. It means that the product was so bad we couldn't even rate it. 1 means the product is pure crap, 2-3 means that the product has a few redeeming qualities, but is not very good. 4 is an okay product, but not something we could recommend. 5 is fair; not great, but not horrible. 6 is a decent product with a few issues, 7 is a strong product with some minor flaws, 8 is a great product with few flaws. 9 is nearly perfect, and 10 (which we haven't handed out yet) is perfect.

As with any big addition to the site, we're really interested to hear what you guys have to say. If you have any input, positive or negative, sound off below in comments!

[TareG Posted Jul 2nd 2010 3:26AM]

Did Engadget give 9/10 for the Retina Display only, or for a device with 2005-dumphone notifications, no widgets, an incomplete web experience (yes, Flash is big and a "small antenna problem" which causes the phone to lose all bars when held?

Did they just do that? Is this a joke? Have we all been punk'd?

Retina Display is awesome (not widescreen, but still awesome) and so is the App Store. But, I mean, with the above mentioned points taken, the iPhone 4 just can't, can't get a 9/10.

[ohnoitsaspider Posted Jul 3rd 2010 1:43AM]

@TareG My mum has a Nokia from 2005 which has an extremely good notification system. Then iPhone can't come close even to that. (The phone also had one-touch video calling over 3G and WiFi. It's a Nokia N81 8GB on Vodafone UK BTW)

[rock0987 Posted Jul 2nd 2010 3:56AM]

Yeah its a sleek and sexy phone , it need some work but overall i feel motorola engineers have done a good job
few new colors would add to the excitement of Android fans
with the Android 2.2
and yeah i would give iPhone 3/10 @ aliendudu5300 , for that .
a phone : the basic part is callig
Android developer

[FerrumMihi Posted Jul 2nd 2010 3:58AM]

Love the idea and the execution is pretty decent. I would like the score cards appear above the articles about the product vai the links

[phearme Posted Jul 2nd 2010 4:51AM]

this is a very bad idea. some devices are not rated the same by everyone because no one has the same needs. for exemple, as an engineer I would rate the Android phones much higher than iPhones, but I understand the people that don't need Android phones.

[Phips Posted Jul 2nd 2010 6:02AM]

I thought this would be a scorecard system we (Us readers) could give on reviewers on Engadget. Oh well...

[indeed Posted Jul 2nd 2010 7:27AM]

Good ol' biased Engadget :D

[spas2k Posted Jul 2nd 2010 8:36AM]

I guess "no 2.2 at launch" (but coming a month or so later) is a much more glaring fault than a phone that drops calls when you hold it "wrong".

[dhamp2g Posted Jul 2nd 2010 2:55PM]

@spas2k Hey man they gave the EVO a 9 and its has andrpiod. The phone is good not great. It's has nothing that other than the processor the specs just fall in line. Oh I love android and apple by the way. And I have an android device as my main phone b4 u call me a apple luvuh.

[paulus Posted Jul 2nd 2010 9:22AM]

The scorecard system is already failing. 7/10 for the Droid X?
That does not line up with the review. Ditto for the iPhone 4.

[Leegh229 Posted Jul 2nd 2010 9:23AM]

iPhone 4 got a nine while droid x a seven... Why not just give the next iPhone a 10 since they are being highly rated on engadget?

[xnox Posted Jul 2nd 2010 9:39AM]

All KIRF get 11/10 !!!!

[2late2die Posted Jul 2nd 2010 9:43AM]

I like the idea of score cards as a quick way to overview the device but I think the numerical score is unnecessary and somewhat counterproductive. Here's the thing, I know you said that the score is "relative to when the review was made" as it were, but people aren't gonna think of that. They're just gonna see the number and immediately make assumptions, and everything (in the review, and the score card) is going to be colored by these assumptions. It doesn't matter if you realize the number is not everything, you see the number and you instantly get a "feeling" about the device.
I think you should get rid of the number and put it in a "3rd section", something like "reviewer's opinion" maybe. Like "if you don't like android this device is not gonna change your mind, but if you're open minded and looking for something versatile this one is something to consider", or whatever - a one/two sentence summary of what the reviewer thought basically.
If you feel the need to have some score as it were, consider stars instead, or a letter grade. While technically they still equate to a number, practically because they're not actually numbers, it's much easier to grasp their "relative to the time of the review" nature.
Consider this, and it's up to each personal individual, but for me, for example, a 7/10 already feels like something I really wouldn't want to buy (and as I said, while I will read the review and form my own opinion, that "7/10 feeling" is gonna color everything. However, a 3.5 stars (out of 5, which would technically equate to 7 out of 10), is something that I would still consider buying. Somehow 3.5 stars feels better than 7/10. (as I said though, maybe it's just me)

[GamerGirl Posted Jul 2nd 2010 10:04AM]

These are wonderful, thank you!!!

[Reggie3 Posted Jul 2nd 2010 12:22PM]

The Droid X being ranked only 7/10 shows some flaws in your algorithm. To rank it bad because it comes with only 2.1 vs 2.2 is not fair unless you can say that a upgrade will not be coming soon.

[danfrumkin Posted Jul 2nd 2010 1:50PM]

You need a Freshness Rating. A review today of 6 of 10, may be much better product that a 1 year old review at 8 out of 10. As the reviews age, they go stale. It would be cool to have the review score stay the same but grow moss or get moldy over time until about 2 years after the original review it just goes black (the value stays there but everyone would know that the device is not likely competitive with today's devices). Just a thought.

[ohnoitsaspider Posted Jul 3rd 2010 3:01AM]

The tm2 gets a 5 but the iPad gets a 9?
what????????????????????