Benchmarking on Trial - Real World vs Timdemos
 

Performance evaluations are never easy and the argument of using synthetic and real world benchmarks is one that's neverending among reviewers, the general public and even manufacturers. However, it is important to realize that there is a difference between benchmarks and reviews - benchmarks is usually a part of a full review, a part that focuses on performance since it is relatively easy to measure than other features of a product (image / sound quality, ease of use and installation etc).

Since most people do put performance first when buying a hardware, it's only natural for reviews to put more focus on performance than say, value added features. Benchmarks are usually done with a fixed setup where only one type of hardware is changed - the hardware that's being benchmarked. As a reference point, a similar product, either in performance, capabilities or price is also benchmarked. Afterwards, the results of the benchmarks are compared to see which one offer the higher performance, according to the benchmark's metrics.

They played through Quake 4 / FEAR / SS2 three times? I haven't even completed a single one of those titles due to lack of time :o