On Sun, May 17, 2009 at 3:28 AM, Scott Ritchie scott@open-vote.org wrote:
Austin English wrote:
Howdy all,
I've been working on the test suite. I've got a few basic tests set up with notepad, and I'm currently working on setting up the framework, using AutoHotKey to both run all the tests and parse the logs for failures/passing todo's.
You may have already had this idea, but today I was just thinking of the concept of running performance tests alongside Wine's conformance tests. Your test suite would be a great place for that - just integrate scripts for the various benchmarking tools. Bonus points if you can put the output from the benchmark programs themselves into some sort of machine parseable format. Then we could have a rough chart of performance data on various things as development continues; in particular we'd catch bugs that still behave correctly but are drastically inefficient.
Possibly, I may look into doing that.
My current plan is to gather all the low hanging fruit. Small, simple applications give us a large range of stuff to test, without adding much difficulty/complexity. Once all that is done, more complex stuff can be added.