benchmark
Latest
Samsung and HTC phones go head-to-head in an 'uncheatable' benchmark test
Remember the allegation that Samsung cheats at benchmarks? Despite the manufacturer's semi-denials, there's growing evidence to suggest that not only Samsung, but also a number of other Android phone makers engage in some level of chicanery. Usually, it involves programming a device to temporarily ramp up its performance if it detects the launch of a benchmark test, regardless of the consequences for battery life or processor temperature. This results in a higher score on the artificial test, but one that is unrepresentative of what the device could actually achieve if it had to pace itself for a real-world task that lasted for a longer period of time -- such as a 20-minute bout of gaming. So far, so bad. However, a startup called GameBench reckons there's another way. Its founders, who previously worked at chip companies like ARM and MediaTek, claim to have developed an "uncheatable" performance test that can be used to corroborate (or refute) the scores from traditional benchmarking apps, and which can help to rank Android phones and devices according to their true gaming capabilities. Although GameBench's app is still in beta and likely won't be released until the first quarter of next year, it has already collected scores for two devices, the HTC One and the Samsung Galaxy S 4. Ironically, as you're about to see, Samsung may actually have less to fear from this cheat-free test than some of its rivals.
Intel Bay Trail benchmarks show big boost for Windows 8.1 and Android tablets
Things are changing at Intel. In the desktop world, the company is used to staggering its efforts with a "tick-tock" product cycle. First it manufactures an existing chip design at a smaller size of transistor ("tick") and then, usually a year later, it improves the architecture while sticking to the same transistor size ("tock"). Bay Trail, by contrast, is a mobile class chip that represents both a tick and a tock. It makes major changes on previous Clover Trail design, while also shifting from 32nm to 22nm transistors. A jump like that holds major promise, so we were keen to benchmark Bay Trail at the earliest opportunity. Intel allowed us into a hotel room in central London to do just that, using the highest-spec quad-core Z3770 chip inside a reference tablet, and the results look impressive. Read on for the stats and a quick assessment of what they mean.
In contrast to Android OEMs, Apple and Motorola don't fudge their benchmarking results
While reviewing Samsung's Galaxy Note 3, reviewers at Ars Technica discovered that the device is configured to artificially inflate benchmark results. We noticed an odd thing while testing the Samsung Galaxy Note 3: it scores really, really well in benchmark tests -- puzzlingly well, in fact. A quick comparison of its scores to the similarly specced LG G2 makes it clear that something fishy is going on, because Samsung's 2.3GHz Snapdragon 800 blows the doors off LG's 2.3GHz Snapdragon 800. What makes one Snapdragon so different from the other? After a good bit of sleuthing, we can confidently say that Samsung appears to be artificially boosting the US Note 3's benchmark scores with a special, high-power CPU mode that kicks in when the device runs a large number of popular benchmarking apps. Samsung did something similar with the international Galaxy S 4's GPU, but this is the first time we've seen the boost on a US device. In the wake of that report, Anandtech found that playing fast and loose with benchmarks is commonplace in the smartphone market, especially amongst Android devices. Indeed, Anandtech found that nearly every OEM engages in similar shenanigans, with Apple and Motorola being the only two exceptions.
Samsung reportedly not alone in cheating Android benchmarks
Samsung has come under fire this week for allegedly inflating the benchmark scores for the Galaxy Note 3 among other Android devices, but -- shocker -- the company is not alone in enhancing scores. AnandTech just published a report that shows ASUS, HTC and other companies using the same "benchmark detect" function to artificially bump up the numbers. According to the site, virtually all OEMs run a CPU optimization on at least one of their devices, save for Apple, Google's Nexus 4 and Nexus 7 and Motorola's latest crop of phones. Dishonesty aside, though, what's the big to-do about these inflated scores? AnandTech found that companies notched less than a 10-percent performance boost in AnTuTu and Vellamo. And when's the last time you based your smartphone purchase on benchmark scores alone, anyway?
Samsung reportedly boosting Galaxy Note 3 benchmark performance by 20 percent
Samsung drew criticism for inflating the benchmark scores of Exynos devices earlier in the year, but the company appears undaunted; it's reportedly boosting test numbers for other hardware as well. Ars Technica has discovered that the Snapdragon 800-based Galaxy Note 3 (and possibly the new Note 10.1) includes code that runs all CPU cores at full speed during certain benchmarks. The tweak gives the smartphone a minimum 20 percent higher score in any affected app, or enough to claim an artificially large advantage over an LG G2 using a similar chip. There may also be a graphics boost, Ars says. We've asked Samsung for its take on the findings. Whether or not the company responds, we don't envy its position -- it's hard to form an alliance devoted to accurate mobile benchmarking when you're accused of doctoring results.
3DMark benchmarking app arrives on iOS, allows cross-platform comparisons
Since there aren't many benchmarking apps for iOS, we totally understand if you resented 3DMark's Android-only release in April. Good thing Futuremark finally launched an iOS counterpart, giving you a way to test hardware and pit the results directly against those of Windows and Android devices. The tool comes with three benchmarking modes that gauge CPU and GPU performance: the basic Ice Storm test, the more demanding Ice Storm Extreme and a new one called Ice Storm Unlimited, which lets you make chip-to-chip comparisons between phones and tablets. 3DMark will work with hardware running iOS 6.0 or higher (except if it's a 4th-gen iPod touch), but you'll need at least 174MB of free space. While this'll surely make a lot of people happy, it's not the end of the tool's foray into new platforms -- Futuremark plans to unleash a Windows RT version in the future.
Windows 8 found to skew benchmark results on overclocked hardware
Overclocking may yield impressive benchmark results, but it turns out scores from Windows 8 PCs may not be reliable. The management at overclocking community HWBOT has discovered that tests provide inaccurate stats when then CPU base clock frequency is fiddled with from within the OS. Hardware-based real-time clocks (RTCs) help keep accurate track of time, but the operating system's timekeeping somehow slows down or ramps up when processing speeds are tweaked. When underclocked by six percent, the outfit's Haswell-infused system lagged 18 seconds behind actual time, fooling the benchmark into a higher score since it seemingly finished in a shorter period of time. Conversely, a boost to CPU speeds results in a lower mark as the internal timepiece ticks away faster than usual. However, modifying processor speeds at boot time avoids these issues. As a result of the revelation, HWBOT is no longer accepting benchmarks from computers running the eighth iteration of Ballmer and Co.'s software, and will invalidate those already in its database. "Simply no benchmark – not even 3DMark – is unaffected by Microsoft's RTC design decisions," the outlet adds. The timing issues are said to stem from Windows 8's support of disparate hardware setups, including embedded and budget PCs that don't have a fixed RTC. If you'd like to see the inconsistencies for yourself, head past the break for video proof.
TUAW test-drives the G-RAID mini portable RAID storage system
G-Technology makes some of the better pro storage products we've tested in the past, and the company provided us with a new 2 TB G-RAID mini (US$449.95) portable RAID system. This is quite the powerhouse for those who need fast, portable storage, including USB 3.0 and FireWire 800 interfaces as well as a pair of 7200-RPM hard drives. Design This is one drive that you won't worry about getting scratched up on the inside of your equipment bag while out on a video shoot. The diminutive drive, which measures just 5.88" x 3.25" x 1.5" (149 x 83 x 38 mm), is wrapped in a practically bulletproof aluminum casing. That aluminum adds a bit of heft to the 2.2 pound (1 kg) case, but helps in terms of dissipating heat through the integrated heat sink. There's also a cooling fan built into the bottom of the case. %Gallery-195941% On the back of the drive you'll find three ports -- two are FireWire 800 ports that can also be used with an included FW 800 to 400 cable with Macs that only support the slower standard, and one is a USB 3.0 SuperSpeed port. For our benchmarks, I used the USB 3.0 port and included cable as my MacBook Pro with Retina display does not include a FireWire port. When using FireWire, the drive is bus-powered; when connected via USB 3.0, it requires the included AC adapter for power. The company also provides a nice leatherette carrying case for the drive. If you don't need a full 2 TB of RAID storage, you might want to consider the 1 TB model that sells for $249.95. The drives can be used in either RAID 0 (striped set for performance) or RAID 1 (mirrored set for redundancy) mode, but realize that your usable storage in RAID 1 will be one-half of the installed storage amount -- in other words, the 2 TB array would be used as two 1 TB mirrored drives in RAID 1. Functionality and Benchmarks When I first took the drive out of its box and connected it to my test machine, it was obvious that another tech blogger had been testing the G-RAID mini. A proprietary bandwidth benchmark was on the drive, so I went into Disk Utility and reformatted the drive. I wanted to test the G-RAID mini in both RAID 0 and RAID 1 configurations, but could not get G-Technology's RAID Configurator application to run without crashing repeatedly on my machine running OS X 10.8.4. The most recent version of this app is three years old; it's time for G-Technology to update it. I tried to set up RAID 0 using Disk Utility, but was unable to do so. If I didn't know any better, I'd think that one of the two drives in the casing had gone bad and I was just seeing one, as the only drive visible in all utilities was just one 1 TB unit. That being said, all results you see here for benchmarks are measuring essentially a single-drive unit. The results of 21 runs of the AJA System Test "Disk Whack" read/write test indicated an average write speed of 125.8 MB/s, with an average read speed of 122.7 MB/s. This benchmark indicates that the G-RAID mini is faster in both read and write operations than two other drives we've tested, a Western Digital WDC1001FALS-40U9B0 that measured in at 90.3 and 94.7 MB/s respectively on write and read speeds, and faster than G-Technology's own G-Drive Mobile at 86.0 and 103.6 MB/s. The G-RAID mini paled in comparison to the Seagate STBV30000100 drive we tested last November, which topped the charts with an average write speed of 171.5 MB/s and read speed of 170.7 MB/s. But for the most part, the G-RAID mini on USB 3.0 is faster than just about all FireWire 800 drives we've tested over the years. Conclusion The G-RAID mini is a sturdy, small and solid RAID array. Unfortunately, it appeared that the unit we tested had only one working drive, and the RAID Configurator app is not compatible with OS X 10.8.4. I would highly recommend that if you're interested in this array, you purchase one and do your own testing. G-Technology's warranty provides a 30-day return policy, provided that you return the device with all cables and in the original packaging in salable condition. Pros Very solid construction Dual interface design with USB 3.0 SuperSpeed and FireWire 800 Quite portable Cons RAID Configurator application is out of date Not bus-powered when connected via USB 3.0 Price is quite high; if you can put up with the extra weight, there are many larger RAID units that are available for much less Who is it for? The video or photography professional who needs a compact and light RAID array for work in the field.
Character creation benchmark now available for Final Fantasy XIV
You don't know when the next (and final) beta test is starting for Final Fantasy XIV. That's a bad thing, because it means you're going through a fair bit of withdrawal. And while nothing can really fix the fact that you want to be playing but can't, being able to play around with the character creator might at least help take the edge off. Which you can do today, as the newest benchmark and character creator is now available on the official site. Aside from offering another chance to test your computer's performance against the game, the new benchmark will allow you to create a new character. It also allows you to save the appearance data from that new character, which can be loaded up again in the launch version or the benchmark for further tweaking. So you could have your character ready to go the moment the game goes live later this month. [Thanks to Brandon for the tip!]
Moto X benchmarks and specs slip out, point to a fine mid-tier phone
We've already seen Motorola's rumored Moto X in press renders, photos, videos and even the hands of a Google executive; why not throw some benchmarks and specifications into the mix? Android Police is more than happy to add that grist to the rumor mill with a set of photos that reportedly show the AT&T Moto X ("Ghost") running AnTuTu and 3DMark tests. If accurate, the scores hint at a mid-range Android 4.2.2 phone whose 1.7GHz dual-core Snapdragon S4 Pro and 2GB of RAM produce good (though not great) results. The rest of the visible specs are equally middle-of-the-road -- we see a 720p screen, 16GB of built-in storage, NFC, a 10.5-megapixel rear camera and a 2.1-megapixel front shooter. The hardware alone may not tempt customers, then, but Motorola could be counting on customization and software tricks to lure more customers. We'll know the full story on August 1st.
iOS 7 Beta 3 gets benchmarked on iPhone 4 and 5
As we inch our way towards the public release of iOS 7, we are treated to a rather interesting benchmarking video courtesy of YouTube user adrianisen. The clip showcases two different benchmark tools running on iOS 7 Beta 3 on both the iPhone 4 and iPhone 5. It's not clear why the tester chose the standard iPhone 4 as opposed to the more powerful 4S, but it certainly makes the performance difference between the two handsets remarkably apparent. An interesting note: Using Geekbench 2, the iPhone 5 used in the test scores a 1,672 with iOS 7 Beta 3, while my own iPhone 5 running iOS 6.1.4 tops out at 1,664 consistently. It's a small increase, sure, but a promising result regardless. The iPhone 4, on the other hand, doesn't even break the 400-point barrier.
Haswell-equipped 15-inch MacBook Pro appears in Geekbench report
It's hardly a matter of "if" -- it's simply a matter of "when." Now that Intel's Haswell CPU has found its way into the MacBook Air line, it's a given that Apple engineers are actively looking for ways to cram that very silicon into the rest of its laptop range. If a newly uncovered Geekbench report is to be believed, it looks as if the 15-inch MacBook Pro could be next in line. The report (pictured in full after the break) details a machine packing a 2.4GHz Core i7-4950HQ and 16GB of RAM, and while there's no discrete GPU shown, Primate Labs' John Poole has informed MacRumors that the benchmark may have simply missed it during testing. At any rate, the score itself isn't anything to write home about -- it's pretty much in line with the existing generation of MBPs -- but the real magic is apt to reside in the eventual battery life claims. If the next-generation Pro follows the Air, we'll see similar performance and nearly twice the longevity. Mission accomplished.
Sony Xperia Z Ultra hands-on redux: benchmark and camera preview
You may have already read our Sony Xperia Z Ultra hands-on last week, but since then we've also been able to spend a tiny bit more time with a pre-production unit (with firmware build 14.1.B.1.277). Instead of going over again how hilariously large this 6.4-inch, pen-friendly phone is, this time we'll focus on some early benchmark results, camera performance and Sony's very own UX features. As you'll see after the break, many of the benchmark scores aren't too far off from what we saw on the MDP phone with the same Snapdragon 800 SoC, and the final units should be optimized with higher numbers. While we didn't manage to get CF-Bench and Quadrant running on the phone, the higher-than-before 3DMark score did cheer us up, meaning either Sony or Qualcomm's managed to fine tune the latter's new Adreno 330 GPU.
Qualcomm Snapdragon 800 MDP benchmarks: prepare for ludicrous speed
Today we had a chance to play with Qualcomm's latest MDP devices (tablet and phone) which pack the company's mighty Snapdragon 800 SoC (MSM8974).
On your mark, get set, benchmark! 3DMark Android Edition now on Google Play
Android just gained another go-to for benchmarking. After failing to hit the 2012 mark for its Android-specific performance software, Futuremark's finally delivering on its promise and making 3DMark available today on Google Play. Typically used as a PC benchmarking tool, the free-to-download app now lets users catalog and compare performance across Windows and Android devices -- iOS and WinRT versions are still listed as "coming soon." There are a few caveats to use, though, as the application requires a smartphone or tablet running Android 3.1 or higher, with 300MB of storage space, a minimum of 1GB RAM and the ability to play nice with OpenGL ES 2.0 (which is about 90 percent of all Android devices, according to Google). Who knows? It could even find a permanent place in our own Android reviews soon. Only time and testing will tell -- check after the break for a video preview of what's in store.
Resident Evil 6 gets a free PC benchmark test
Resident Evil 6 drops on PC on March 22, and if anyone is still interested in playing the game and attempting to have a good time, there's a free benchmark test available now.Capcom announced the benchmark test in January, alongside news of a new mode called "The Mercenaries: No Mercy." It's based on the existing Mercenaries mode for consoles, but with "more enemies on screen than ever before." Ada Wong's campaign also comes standard with the PC version of Resident Evil 6.If that all sounds wonderful, see if your rig is up to snuff with Nvidia's free PC test.
NVIDIA Tegra 4 benchmarked, breaks all sorts of speed records (video)
When NVIDIA unveiled Tegra 4 back at CES, we scrambled to get hands-on with a reference device. And though our initial performance impressions were positive -- it runs 1080p video and games smoothly -- there was only so much we could say to illustrate how fast the performance is. After all, Tegra 3 already does a fine job handling games and full HD movies. What we really needed were benchmarks, some quantitative data to help show the difference between Tegra 4 devices and whatever's currently on the market. Fortunately for all of you, we just got our chance: here at Mobile World Congress, the company has reference tablets set up expressly for the purpose of running tests. So, we did just that... over and over and over until we had a long list of scores. Meet us after the break to see how it fared.
Primate Labs benchmarks new Retina MacBook Pro models
Apple bumped the processors in the Retina MacBook Pro line last week, pushing the 15-inch model to the 2.4, 2.7 or 2.8 GHz quad-core Intel Core i7 and the 13-inch to 2.5, 2.6, 2.9 or 3.0 GHz dual-core processors. As usual, Primate Labs grabbed a handy copy of their "flagship product" -- Geekbench -- and tested each model's performance. The 100 MHz increase in processor speed across the board on the 15-inch MacBook Pros was responsible for a 3 to 5 percent increase in performance as measured by the Geekbench 2 scores. Likewise, the 13-inch models received a bump in both speed and performance. Primate Labs noted, though, that the 13-inch Retina MacBook Pro models ship with both old and new processors. The new processors are only available in 13-inch models that start with 256 GB of storage, while the machines that start with 128 GB of storage retain the older, slower processors. John Poole at Primate Labs notes that the new Intel processor architecture, Haswell, is forthcoming later this year, so it's not surprising that Apple decided to outfit the Retina MacBook Pro line with the latest in Ivy Bridge processors. However, he's curious about why the non-Retina MacBook Pro line didn't receive the same speed bump, and speculates that the current line of Retina-free MBPs might be the last.
New 3DMark arrives for Windows, gaming PCs brace for punishment (video)
There's nothing quite so crushing to the self-esteem of a gamer as a new 3DMark build: even a year-old flagship PC can feel like a clunker as it struggles to get a decent frame rate in Futuremark's visually intensive tests. It's time to get masochistic once again, as the Windows version of the 2013 3DMark release is at last available. The software's showcase benchmark is Fire Strike, for brag-worthy rigs with high-end DirectX 11 graphics; there's also a Cloud Gate test for mere mortal DX 10/11 PCs, and Ice Storm for older DX 9 hardware. 3DMark is waiting both as a stand-alone download and through Steam today, while those who take their gaming on the road will have to wait for the still-in-progress Android, iOS and Windows RT releases to make their tablets cry for mercy.
'Epic Citadel' tech demo now on Android
Android users still can't partake in the acclaimed slashing hijinks of Epic Games' mobile showpiece Infinity Blade or its sequel, but they can walk around its medieval setting and watch things, like, render and stuff, now that Epic Citadel is available for free on Google Play and the Amazon Appstore.Originally released on iOS way back in the stone ages (read: September 2010), Epic Citadel serves as both Unreal Engine 3 tech demo and phone hardware benchmarking tool, allowing Android users to not only see how pretty UE3's rivers are, but also how well their devices handle rendering them. Our Galaxy Nexus, for instance, ranked in at the "High Performance" level, churning out an average of 43.9 FPS at 1196 x 720. During testing, framerates peaked just below 60 FPS and momentarily bottomed out around 18 FPS.As with the iOS release, there isn't a tremendous amount to actually do in Epic Citadel, aside from looking at fluttery banners and lens flares, and its on-screen twinstick controls are only recommended for those who enjoy being 100 percent out of control of their movement. Still, it's very pretty. %Gallery-177522%
