AnandTech alleges MediaTek cheating on benchmarks
 

Helio P95 outperforming Dimensity 1000L

AnandTech claims to have caught MediaTek fudging its benchmarks.

In a long and rambling article where the writer spends rather too much time talking about himself, Andrei Frumusanu said that he smelt a rat when he had first received Oppo’s new Reno3 Pro – the European version with MediaTek’s Helio P95 chipset.

“The phone surprised me quite a bit at first, as in systems benchmarks such as PCMark it was punching quite above its weight and what I had expected out of a Cortex-A75 class SoC. Things got weirder when I received a Chinese Reno3 with the MediaTek Dimensity 1000L – a much more powerful and recent chip, but which for some reason performed worse than its P95 sibling. It’s when you see such odd results that alarm bells go off as there’s something that is quite amiss.”

It appeared that the chip was engineered to recognise a benchmark test and give an excellent result to match the test.

Frumusanu alleged that the cheating mechanism had seemingly been sitting in plain sight to users for several years:

In the device’s firmware files, there’s a power_whitelist_cfg.xml file, most found in the phone’s /vendor/etc folders. Inspect the file, and there is a list of popular applications with various power management tweaks applied to them and a list of multiple benchmarks. We find the APK ID for PCMark, and we see that some power management hints are being configured for it, one common one is called a “Sports Mode”.

"The benchmark list here isn’t exhaustive. But it contains the most popular benchmarks in the industry today – GeekBench, AnTuTu and 3DBench, PCMark, and some older ones like Quadrant or popular Chinese benchmark 鲁大师 / Master Lu. There’s also a storage benchmark like AndroBench2 which is a bit odd – more details on that later."

“Sports Mode” fixes some DVFS characteristics of the SoC such as running the memory controller at the maximum frequency all the time. The scheduler is being set up to be a lot more aggressive in its load tracking – meaning it’s easier for workloads to have the CPU cores ramp up in frequency faster and stay there for a more extended period, applying a few common boosting mechanisms.

https://fudzilla.com/news/50625-anan...-on-benchmarks