Patch checking robot coming

[Dan Kegel]

After some discussion with Jeremy and Alexandre
about how nice it would be to have some automated
way to check patches before Alexandre commits them,
I decided to have a go at creating one.

What I have so far is a script that watches wine-patches
and applies each patch to current git, then builds,
and sends email with a subject line saying whether
it worked, and a body containing a short "I'm a build robot"
note, the patch, and the build log.

You can see the code at
http://code.google.com/p/winezeug/source/browse/#svn/trunk/patchwatcher
Its main failing is that it doesn't understand patch
series yet, but I should be able to get that going
by rewriting get-next-patch.pl to take a bit more context into account.

Right now it only emails the results to me, but it's probably
nearly good enough to show results to other people.

Question: where should the results go?
I could put them up on a web site
and/or email them to the patch's author
and/or email them to a dedicated mailing list (e.g.
[e-mail]?)
and/or email them to wine-devel.
- Dan

[Ambroz Bizjak]

Dan Kegel wrote:
> What I have so far is a script that watches wine-patches
> and applies each patch to current git, then builds,

Just where are you going to run that? To me, a script that builds just
every patch is a serious security flaw; I suppose it wouldn't be very hard
for someone to send a naughty patch that would take control of your
machine. Something like editing a Makefile. I suggest you make it build
patches in a chroot as a regular user, and copy over the chroot from a
template every time a new patch is being built.

[Francois Gouget]

On Sat, 2 Aug 2008, Dan Kegel wrote:
[...]
> Question: where should the results go?
> I could put them up on a web site
> and/or email them to the patch's author
> and/or email them to a dedicated mailing list (e.g.
> [e-mail]?)
> and/or email them to wine-devel.

I'd send the results to:
 * a website so everyone can see the status of each patch and how 
   backlogged the bot is (especially when people send a series of 30 
   patches at once).
 * to Alexandre to use as a replacement for wine-patches so he'd have 
   more information about each patch.
 * to the submitter but only when the checks failed.
   The website could potentially let you tune this in your preferences
   (never receive any email, receive both success and failure).

A dedicated mailing list could work too but I'm not sure there's really 
a point.

Submitters are just interested in their patches and it seems easier to 
find out about their status directly via email, or by looking on the 
website.

The dedicated mailing list could be useful to reviewers in that it would 
let them focus on just the patches that have been vetted by the bot. But 
I'm not sure that would be such a good thing. The other submitters may 
need guidance too...

[Dan Kegel]

On Mon, Aug 4, 2008 at 2:07 AM, Francois Gouget  wrote:
> I'd send the results to:
> - a website so everyone can see the status of each patch and how
>
> backlogged the bot is (especially when people send a series of 30
>   patches at once).

Yes.
> - to Alexandre to use as a replacement for wine-patches so he'd have
>
> more information about each patch.

If he asks for it.  (Maybe he'll be happy with the web site.)
> - to the submitter but only when the checks failed.

Yes (that's how the code is now).
> The website could potentially let you tune this in your preferences
>   (never receive any email, receive both success and failure).

Yes.  I'll start off with static html, the dynamic part can be
added later if it all seems worth it.
- Dan

[Alexandre Julliard]

"Dan Kegel"  writes:
> > - to Alexandre to use as a replacement for wine-patches so he'd have
> >
> > more information about each patch.
>
> If he asks for it.  (Maybe he'll be happy with the web site.)

what should really happen is that patches that fail the validation don't
even make it to wine-patches, they get bounced to the submitter. Of
course the bot needs to be reliable enough for this not to become a
nuisance. Maybe there can be a magic bypass keyword to add to a patch to
override the bot.

[Dan Kegel]

On Mon, Aug 4, 2008 at 5:09 AM, Alexandre Julliard  wrote:
> what should really happen is that patches that fail the validation don't
> even make it to wine-patches, they get bounced to the submitter. Of
> course the bot needs to be reliable enough for this not to become a
> nuisance. Maybe there can be a magic bypass keyword to add to a patch to
> override the bot.

That can be arranged.  First things first though,
let's get a simple bot working off to the side;
then once it's reliable enough, we can insert it
into the normal workflow.
- Dan