GPU Benchmarking Methods Investigated
 

Let’s begin with the most popular way of benchmarking games: in-game or stand-alone rolling benchmarks that spit out a result with minimal user involvement. In general, they don’t mean jack all when it comes to determining a GPU’s performance within the game itself. Stand alone benchmarks are heavy offenders for a number of reasons including a lack of patches and sequences which aren’t at all representative of in-game conditions. In-game rolling benchmarks receive all of the necessary game engine patches but more often than not still fall short when it comes to displaying actual gameplay. However, there are currently a small number of games like DiRT 2 and HawX which incorporate benchmark sequences that accurately recreate in-game scenarios. In this category we believe stand-alone benchmarks should be avoided altogether while in-game benchmarks should only be used if they represent actual gameplay and don’t include a “flythrough”.

http://www.hardwarecanucks.com/forum...s-fiction.html