Fudging benchmarks? So what benchmarks still suck

adb shell cat /sys/module/pvrsrvkm/parameters/sgx_gpu_clk can never tell a lie

Samsung got caught cheating. It is manipulating benchmark results for the Galaxy S4by allowing the GPU to run at a higher clock speed and locking the CPU to the max frequency, in a different way than other system processes are allowed to do. They give a few of Samsung's own apps limited time with these higher speeds, but clearly favor some popular benchmark apps. They got caught red-handed by Brian and Anand over at AnandTech two of the best hardware geniuses in the business and the Internet is in a tizzy, as the Internet is wont to do.

We've been reading all about it, including the responses of people who are in no way affiliated with any of the involved parties, and now that it's past noon and I've had a few cocktails with lunch I want to write a few words about it.

Join the fray in the Galaxy S4 forums: Samsung cheats at benchmarks

To make things better (or something), Samsung released a statement clarifying a few things.

Under ordinary conditions, the Galaxy S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode. Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance.

The maximum GPU frequencies for the Galaxy S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results.

We remain committed to providing our customers with the best possible user experience.

The problem is, that's not quite true.Anandtechhas data that shows Samsung isn't being completely truthful especially the part where it claims things "were not intended to improve certain benchmark results." It also completely glosses over the part in the code where "BenchmarkBooster" actually names a few popular benchmarking applications that are able to set a special "boost_mode" flag when they are launched.

Maybe this statement came from someone who wasn't familiar with the situation, and it's an honest mistake. That doesn't mean it's not false. And people are going to rightly jump all over Samsung for saying it.

But it doesn't really matter.

Benchmarks suck. I know benchmarks suck, because I have a Galaxy S4 and a Nexus 4 here beside me, and the scores in no way reflect the user experience. They are just numbers for people who like to look at numbers and tell other people that their numbers are bigger than your numbers, or something like that. When I want to know how well any computing device, not just an Android, handles the system and apps we like to install, I turn it on and use it. Something like "25,611" means nothing to me. The fact that your phone got "25,612" also means nothing to me. In fact, we only run benchmark apps when our inboxes won't shut up and stop asking us to run them, and even then only in a momentary lapse of reason. Because they suck.

They're also super easy to manipulate, as evidenced by Samsung's latest gaffe as well as shown here. If you want numbers, just tell me what number you want and we'll make it happen. You still will have no idea what they mean in the real world and how it matters when you want to play Words with Friends of stream porn from the Internet.

Kudos to Anandtech for digging to the bottom of this, and shame on Samsung for playing into the game and then giving a wishy-washy response about it when they got caught. That needs said, so Ill say it. But as for benchmarks themselves, and whatever numbers Samsung's Galaxy S4can turn in, I don't care, and I don't think you should care either.

Via: Fudging benchmarks? So what benchmarks still suck