Related Articles
Much thought goes into evaluating the quality of a given tablet here at CNET. From the way it feels in your hands, to the speed in which you can access your favorite Web site, to the frame rates you can expect when playing games, in the end, all the pieces matter. And while it’s difficult to empirically evaluate the feel of a tablet, gauging its performance is a different story altogether.
What follows is our current tablet testing procedure, which consists of a battery life test, and display performance evaluations, plus a few anecdotal speed tests. Expect these tests to continually change and evolve as we’re currently looking at a few other benchmarks like 3DMark to add to our benchmarking methodology. 3DMark will be compatible with iOS, Android, and Windows RT, making it a very exciting prospect for evaluating 3D games performance. Look for an update whenever the new benchmark is released.
Anecdotal evaluation
We evaluate each tablet’s build quality, taking into account many attributes. These include, but aren’t limited to, the following: its weight, how durable it feels, and whether it’s too malleable. Does it feel overly plastic-y, or does its metal backside weigh it down too much? We also launch plenty of apps to get a feeling for how fast the tablet is at performing obviously important tasks. We give the Web a spin through the tablet’s native browser, taking into account how fast our favorite sites load.
Screenshot by Eric Franklin/CNET
One of our favorite anecdotal tests also is one of the most useful. Riptide GP is a polygonal watercraft game that’s ideal for testing tablet GPU performance, and depending on the speed of the tablet’s processor, the game’s frame rate will be noticeably smoother or choppier. On the Android and iOS versions of the game, the resolution is also scalable to the tablet screen’s native size. Unfortunately, there’s no frame-rate counter built into the game, so its smoothness and clarity must be determined subjectively.
Contrast ratio and brightness
We test the maximum brightness, default brightness, and contrast ratio for each tablet. We conduct these tests using the Minolta CA-210 display color analyzer.
Eric Franklin/CNET
We use a completely white screen to test the brightness and a completely black screen to test the black level. With each screen at full luminosity, we place the sensor in the middle of the screen to gather a reading. We then divide the maximum brightness by the maximum black level to get the contrast ratio.
Battery life
We evaluate battery life by continually running a movie file on the tablet until its battery dies. We set each tablet to Airplane mode and adjust its respective brightnesses to 150 candelas per square meter (cd/M2) or as close to that number as possible.
For iPads, we run the iPad version of “Toy Story 3.” On Android tablets, we run a 720p version of “Toy Story 3.” The reason we chose 720p for Android is that not every tablet can run 1080p video just yet, and we want to make sure we test all Android tablets under the same methodology. Look for this to change soon as more and more 1080p-capable tablets are released. For Windows RT tablets, we run a 720p version of “Toy Story 3” as well.
Eric Franklin/CNET
On the iPads, we run the movie through an app simply called Video app; for Android, we used the movie player apps mVideoPlayer or DicePlayer, as they provide a much-needed repeat video function that not all native Android movie players include. Also, thanks to its repeat function, we use the app Multimedia 8 on Windows RT tablets.
We’re currently unable to sideload apps onto the Nook HD or HD+. As such, Nook tablets cannot currently use video player apps with repeat functions. Instead we use the Nooks’ native video player and run the HD version of “The Avengers”, restarting the movie each time it gets close to the end. We do this until each tablet’s battery dies.
As I said before, look for tablet testing to continually evolve over the next few months.
