Benchmarking tool for reviewers / diehard overclockers
 

I'm currently starting with a project and i'll need some [M]ad input. :)

I'm going to write a program that will make it much more pleasent for us to do some extensive benchmarking. Anyone who has written a review of motherboard/cpu/vidcard comparison knows that 80% of your time is consumed by running a benchmark program, writting down the results, importing them to an excell sheet and turning it into a good looking graph. This is a very timeconsuming and repetetive task, which can be automised.

My goal is to write a little app, that will allow you to select a few popular benchmarking tools (eg. UT2003 bench, Superpi, 3Dmark, ...), starts the first automaticly, extracts the results when done, closes it and starts with the next one. When all selected benches are finished, all the data is put into a snapshot-ready graph & into a text file (with delimiters, ready to be imported by excell). Saved results can be compared to other results, to make comparisons between different mobo's/vidcards/... easy.

In a nutshell:
- select the benchmarks you want to run
- select options (eg. 3D in low res to test CPU/Mobo)
- start. The app will run the selected benchmarks in sequence and extract the results.
- join the results in a premade graph (for quick screenshot) or output to a excell compatible .txt file for import.

That's it. It would be opensource java.

What i need from you guys?
* would you use (it if it ran perfectly)?
* which benchmark progs should be included? Remember the results have to be extracted somehow, this will not be easy in some cases. Also, there should be a way to start it immediatly (and not wait for user input).
* what kind of output other than specified do you want?
* other comments :)


Current version: MadMark alpha v0.01
-------------------------------------------------------
- initial release
- no export, only pifast and 3Dmark01

Planned bugfixes for next release:
--------------------------------------------------
- app doesn't respond after pifast fixed 11/06 :)
- automatic copy of needed 3Dmark files fixed 11/06 :)
- correct dir check fixed 11/06 :)


Next release will be published asap, i'm just going to test it a few more times.

* Hell yeah I would use it!
* Serious Sam / Q3A (qbench springs to mind) / UT2003 / 3Dmark2k1 / SuperPi / Sisoft Sandra / Aquamark v2 (3 upcoming) / WinAce .. and more =)

Serious Sam => difficult to extract the results, are the average fps saved in a seperate file?
Q3A (qbench springs to mind) => same as SS
UT2003 => no prob
3Dmark2k1 => need to look into that
SuperPi => no prob
Sisoft Sandra => hmmm... dunno yet
Aquamark v2 (3 upcoming) => same as SS
WinAce => no prob

Qbench gives .txt in the end

sounds like a great idea !

but hey, 3Dmark2003 is missing on the list...

and er, how do think to implent the options of 3Dmark like AA-> on/off in that program? (just thinking of the practical realisation)

True, that will be the hardest to implement. For starters, i don't know if you can start 3Dmark without having to click "benchmark".

I'll start with the easy programs first (those who can be started with one simple command and have their results set in a .txt file, eg. pifast), and move on to the harder benchmarkprogs later on.

alright, i've read the helpfile and it seems like 3D mark 2001 / 2003 will be no problem. :)

are you that good in programming ?

in what language are you going to write it ? C ?

Re: Benchmarking tool for reviewers / diehard overclockers
 

:)

So basically you would have to shut down all programs which may affect the bencmarks just like you do whrn, running 3DMark2003/2001SE, Serious Sam, Code Creatures, etc. Hor example adjusting disply options to diable vsync, and 1024x768, and just lert it run thorugh all the programs? But what if say you wanted to run Return to Castle " in 1200x1600 as well? Would you be able to program the sotware to make those display adjustments? I'm not asking for everything, I'm just curious how far you were going to take it? Man you could sell the program to review sites? Or will it be freeware? If so thank you, either way thank you. I think it'll be very useful, although as benchmarks become unpopular, will the program be capable of having it's parametters changed in that respect?

That's the main idea. As the program is idle during benchmarking, and is pretty small in size, i think it won't have a noticable effect on the performance.

-edit-
Pifast (10milion digits) confirms it: difference between a normal run, and run from within the benchmark tool is 0.07s.

Depending on the benchmark, it's possible. For example, it's perfectly possible to launch 3D mark in different resolutions, but i don't know whether this is possible with every game/app. I think the first version of the program i will only include settings to test cpu/motherboard (3D benches at low res) and one to test the vidcard capabilities (3D at 1024*768).

If ppl like the program, i'll try to include several options to bench (low/high resolution, FSAA, Antisoptric filtering).

Ter Angreal had a nice set of Windows tweaks which i'll try to implement too.

After the first working version is finished, it will be open source java. It's free. :)

Java allows programmers to make a very flexible program. If i can it working as I've got planned, adding a new benchmark to the tool will be a child's play.

all hail RB :ws:

Small update:
3Dmark03 works, but the professional version will be needed. This won't be a problem for most reviewsites / serieus benchers i hope. Will try to work around it later on.
Pifast: works
Superpi: seems te be impossible to integrate in the tool :/

Edit:
I have a GF4 MX440, so i can only run the first game test. Would someone with a Dx9 vidcard & 3Dmark03 PRO launch 3Dmark with the option "-verbose" ("3dmark03.exe -verbose") and send me the verbose***.txt file it has generated after running a FULL benchmark? Thx :)

edited

edited post

i think you're command 'verbose' is incorrect

what do you mean with your post calantak

Stop editing your posts damnit, i'm curious. :)

The -verbose option ONLY works if you got the full (registerd)version of 3Dmark03.

i have a full registered version
and it doesn't work

pay attention under the 3dmark - there is proffesional

You've forgotten the quotes. :)

i don't have you're mail adress so i posted it on my site

http://users.pandora.be/kipni/pc stu...ose_JEROEN.txt

thx!

kudos I give you a lot of credit for writing this software. It's amazing how some of the most inventive people, are willing to share their ideas freely, and the fruit of their labors. There's some serious good Karma found in certain forums. Thanx RichBa5tard.

it's starting to get shape :)

improved the GIU a bit and made options possible:

nice work.

indeed nice work
when will the test be available ?

june 23the at the last
we have our exam visual graphic systems(read:java with a lot of swing classes)
he has to present this proggie then.

exactamundo ;)

how's your app going? Any ideas yet?

my app is going nowhere for now
had an idea during OS-exam(besturingssystemen)
it was some OC-game
you have to push buttons to cool/softer harder ,up/lower volt ,up/lower fsb en up/lower multi
and reach a score in somthing foldingalike
but when you push too hard you crash and you lose.
the main thing is it'S like holding several manuals like in Tony hawk pro skater for a certain period.
but dunno if i'll ever get it to work
if it's not clear just know that i'm not a genius .

Small Q, is it already possible to run 3D mark 2001 on a DX9 system ??

yes it works perfect :)

I there a patch for DX or 3dm, cause I'm getting the same error all over again.
you must install DX8.1 to run this version of 3dm or whatever.

3dmark2001 works flawlessly over here with dx9 :)

does anyone care to betatest?

signing up for beta duty:)

me want to be bètatester
can i can i ? :^D

I'd like to volunteer, if it would help.

alright, i'll mail you three a download link to test it in a jiffy. All you need is 3Dmark or Pifast installed, and Java Virtual Machine (you probably have it installed, but if not: download JVM at java.sun.com)

is it just java plugin you have to have installed?

when will you send a mail?

thx for the effort!!!

greetzzz