A script for automatic regression testing

[Mikolaj Zalewski]

I wrote a small script that automates regression testing. It requires 
an Autohotkey script that tests the program and creates a file 
C:\Test-failed.txt if the test failed. The bash script will then do the 
regression testing. It starts with copying the source tree and 
wineprefix so it should be possible to do some normal work during 
compilations.
At the beginning of the script there are some variables that needs to be 
configured before running it. The wineprefix has to have Autohotkey 
installed and associated with *.ahk. The attached script's settings will 
find the regression for bug #9249.
If anyone is interested in it I'd be interested in some feedback.

Mikołaj Zalewski

[Vit Hrachovy]

Hi Mikolaj,
see winebot - a console Wine package tool for Windows programs for how 
to automate AutoHotkey installation for given wineprefix.

http://winebot.sandbox.cz

Regards
Vit

Mikolaj Zalewski wrote:
> I wrote a small script that automates regression testing. It requires 
> an Autohotkey script that tests the program and creates a file 
> C:\Test-failed.txt if the test failed. The bash script will then do the 
> regression testing. It starts with copying the source tree and 
> wineprefix so it should be possible to do some normal work during 
> compilations.
> At the beginning of the script there are some variables that needs to be 
> configured before running it. The wineprefix has to have Autohotkey 
> installed and associated with *.ahk. The attached script's settings will 
> find the regression for bug #9249.
> If anyone is interested in it I'd be interested in some feedback.
>
> Mikołaj Zalewski

[Jakob Eriksson]

Completely awesome, thank you!

Mikolaj Zalewski wrote:
> I wrote a small script that automates regression testing. It requires
> an Autohotkey script that tests the program and creates a file
> C:\Test-failed.txt if the test failed. The bash script will then do
> the regression testing. It starts with copying the source tree and
> wineprefix so it should be possible to do some normal work during
> compilations.
> At the beginning of the script there are some variables that needs to
> be configured before running it. The wineprefix has to have Autohotkey
> installed and associated with *.ahk. The attached script's settings
> will find the regression for bug #9249.
> If anyone is interested in it I'd be interested in some feedback.
> Mikołaj Zalewski

[martin pilka]

Hi Mikolaj,
nice work!

I saw that you successfully ran cxtest.sh script on your machine and
submitted results. Currently, we are in process of setuping cxtest.sh
(tests Wine make test, Picasa, WinZip, WordViewer, ExcelViewer, PptViewer)
and 3dMark2000 tests nightly on our testing machines. Once we have this, we
will need functionally equivalent to script you wrote, which would help us
to find defective patch for regression cases like this:

http://www.cxtest.org/product-evaluation?id_product=3&id_failure=170&...

Would you be interested in joining forces, working together and integrate
your script into CxTest backend?

Martin

[Mikolaj Zalewski]

Currently I'm trying to learn how cxtest works. But it looks like a 
good idea to find sources of such regressions. I could help to write 
such a script. I will check how my script fits into it.

Mikolaj Zalewski