Digit-Life Condems FRAPS 3D Performance Testing

Let's draw a bottom line of another FRAPS research. Fourfold testing on each computer without processing results is too rough, and its measurement error may reach 10-15%, which is absolutely unacceptable! Even though the frame rate in many games is relatively stable and the graphs show similar peaks, the difference in the average FPS values is sometimes too big. It makes no sense to compare performance of graphics cards using such tests, as performance differences between some cards may sometimes be less than 10%. Even if you carry out 4-10 tests, you are careful with the test procedure, and you process results by discarding anomalous readings, you cannot be certain about the resulting figures.

To say nothing of the huge difference between the highest and the lowest FPS results. Games generate slightly different situations each time, you cannot get identical 3D scenes. For example, the frame rate in many racing games is affected by many things: changing weather, time of the day, traffic density (if it exists), changing behavior of computer players, skills of a player. Each time a tester is presented with a different situation, so the frame rate differs much as well. In this case the only solution is to run multiple tests and discard anomalous results.