Re: Buildfarm status monitoring (was Re: pgsql: Implement channel binding tls-server-end-point for SCRAM)

On 01/08/2018 11:01 AM, Tom Lane wrote:
> Simon Riggs <simon(at)2ndquadrant(dot)com> writes:
>> On 4 January 2018 at 21:02, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
>>>> Implement channel binding tls-server-end-point for SCRAM
>>> Buildfarm doesn't like this one bit.
>> Can't we automate these messages?
> It's not that easy. First, the buildfarm gets random failures all the
> time, due to this and that. Second, if several commits have occurred
> since the critter's last run, it requires some human judgment to figure
> out which commit is probably to blame.
>
> You could ameliorate the first problem by waiting for multiple failures
> to show up ... but the longer you wait, the worse the second problem
> becomes (and the less useful the report would be anyway).
>
>> It would be useful to get automatic message giving a summary of
>> buildfarm results at 15, 30 and 60 minute intervals, even if it is
>> just ALL CLEAR.
> The raw result of that would be too noisy to be useful. I've wondered
> about getting the buildfarm status page to filter out the more obvious
> classes of "random failure" --- git pull failures would be one, and
> another would be if "no space left on device" appears anywhere in any
> of the report's log files. Don't know how far that would get us, though.
>
>

Without triangulating via something like git-bisect I suspect we'd very
soon find any automated system very tiresome indeed.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services