Re: Buildfarm feature request: some way to track/classify failures

Andrew Dunstan wrote:
> Martijn van Oosterhout wrote:
>> On Tue, Mar 20, 2007 at 02:57:13AM -0400, Tom Lane wrote:
>>
>>> Maybe we should think about filtering the noise. Like, say, discarding
>>> every report from mongoose that involves an icc core dump ...
>>> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=mongoose&dt=2007-03-20%2006:30:01
>>>
>>>
>>
>> Maybe a simple compromise would be being able to setup a set of regexes
>> that search the output and set a flag it that string is found. If you
>> find the string, it gets marked with a flag, which means that when you
>> look at mongoose, any failures that don't have the flag become easier
>> to spot.
>>
>> It also means that once you've found a common failure, you can create
>> the regex and then any other failures with the same string get tagged
>> also, making unexplained ones easier to spot.
>>
>>
>>
>
> You need to show first that this is an adequate tagging mechanism, both
> in tagging things adequately and in not picking up false positives,
> which would make things worse, not better. And even then you need
> someone to do the analysis to create the regex.
>
> The buildfarm works because it leverages our strength, namely automating
> things. But all the tagging suggestions I've seen will involve regular,
> repetitive and possibly boring work, precisely the thing we are not good
> at as a group.

this is probably true - however as a buildfarm admin I occasionally
wished i had a way to invalidate reports generated from my boxes to
prevent someone wasting time to investigate them (like errors caused by
system upgrades,configuration problems or other local issues).

But I agree that it might be difficult to make that "manual tagging"
process scalable and reliable enough so that it really is an improvment
over what we have now.

Stefan