Jakob Eriksson wrote:
Alexandre Julliard wrote:
If we require tests to pass on all Windows versions before getting committed it will drastically reduce the number of tests accepted, with little benefit. In most cases tests fail on some Windows boxes because they are too strict in the behavior they expect, and that's not a problem for us.
Except that the tests clutter up the reports. We should have at least one dedicated, declared sane, Windows installation that we regard as most important. When you _start_ expecting tests to fail is when you _stop_ paying attention to tests.
(That Codeweavers do not have such an installation yet, is beoynd me. Or if you do, please make it automatically submit its findings to test.winehq.org!)
We do. I've got a machine that regularly runs the test on Windows 2003 on real hardware: http://test.winehq.org/data/200710241000/2003_rshearman/report
However, the tests are run by a service rather than manually by me to reduce the effort needed. This results in some tests failing when they perhaps shouldn't.
This also doesn't help the Direct3D developers since D3D is disabled by default on Windows 2003, so we also need a real Windows XP box to run the tests regularly.