Re: Automatic testing of patches in commit fest

On 09/11/2017 03:01 PM, Aleksander Alekseev wrote:
> Hi Tomas,
>
>>> Unless there are any objections to give this idea a try I'm willing to
>>> write and host a corresponding script.
>>>
>> That won't work until (2) is reliable enough. There are patches
>> (for example my "multivariate MCV lists and histograms") which
>> fails to apply only because the tool picks the wrong patch.
>> Possibly because it does not recognize compressed patches, or
>> something.
>
> Agree. However we could simply add an "Enable autotests" checkbox to
> the commitfest application. In fact we could even left the
> commitfest application as is and provide corresponding checkboxes in
> the web interface of the "autoreviewer" application. Naturally every
> automatically generated code review will include a link that
> disables autotests for this particular commitfest entry.
>

I think we should try making it as reliable as possible first, and only
then consider adding some manual on/off switches. Otherwise the patches
may randomly flap between "OK" and "failed" after each new submission,
disrupting the CF process.

Making the testing reliable may even require establishing some sort of
policy (say, always send a complete patch, not just one piece).

>
> I hope this observation will change your mind :)
>

Not sure. But I'm mostly just a passenger here, not the driver.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services