Okay, so I got winetest to do something rational, and now I'd like to see if it will work for someone else.
It's easy!
Just apply this patch to a current Wine tree: http://www.winehq.org/pipermail/wine-patches/2008-February/049675.html
Then configure and make Wine as you normally would. Then, cd programs/winetest ./runtests <your-tag-here> (The tag I used was jwhite-etch64; I think you should identify who you are, and then if you have multiple systems, your system within that).
If you have winetricks in your path, it should be completely unattended. (If you don't, you'll need to hang around to click 'go' to the Gecko install dialog).
After about 5 minutes, your test results should be run and sent up to test.winehq.org. The cron job runs every 5 minutes, so sometime after that, a reload of: firefox http://test.winehq.org/data/%60date +%Y%m%d`/
Should show you your results! Easy as Pi!
There is still quite a bit to do. First, I'm waiting for a 'real' winetest expert to wack me with a clue bat. But I'll stumble along until the hammer falls...
Second, I want to do a stupidity check and make sure that winetest essentially correlates to a make -k test run. If they are not really identical, then we need to fix that problem...
Next, I think we should probably shift the build-id from being the date to being the HEAD sha1, but that's a radical enough change that I felt I needed to start simpler and within the current conventions.
But I'd really appreciate it if folks could try this and see if we can gather some data quickly. In theory, we should be able to quickly see the common points of failure and know where to attack first.
Cheers,
Jeremy