From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> |
Cc: | Robert Haas <robertmhaas(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Noah Misch <noah(at)leadboat(dot)com>, Clément Prévost <prevostclement(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: parallel.c is not marked as test covered |
Date: | 2016-06-20 22:50:14 |
Message-ID: | 5840.1466463014@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
I wrote:
> Depending on what the percentage actually is, maybe we could treat
> this like the "random" test, and allow a failure to be disregarded
> overall? But that doesn't seem very nice either, in view of our
> increasing reliance on automated testing. If "random" were failing
> 90% of the time on some buildfarm critters, that would probably
> indicate a real problem, but we'd likely not realize it for a long time.
BTW, this is getting a bit off-topic for this thread, but: I got
curious about whether any such effect might actually exist, and decided
to grep the buildfarm logs to see how many times a failure of the
"random" test had been ignored. The answer is that there are 35 such
occurrences in otherwise-successful buildfarm runs, out of something
like 650000 successful runs in the buildfarm database. This is in the
noise, really. There are more occurrences than that of cases where
"random ... failed (ignored)" appeared in a failed
run, which more than likely means that some other regression test script
caused a crash with collateral damage to this test.
This seems like pretty good evidence that we should remove the "ignored"
marking for the random test, and maybe remove that functionality from
pg_regress altogether. We could probably adjust the test to decrease
its risk-of-failure by another factor of ten or so, if anyone feels like
0.005% failure probability is too high.
Raw data attached for amusement's sake.
regards, tom lane
Attachment | Content-Type | Size |
---|---|---|
random_ignored.results | text/plain | 9.8 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Thomas Munro | 2016-06-20 22:59:25 | Re: Reviewing freeze map code |
Previous Message | Peter Geoghegan | 2016-06-20 22:38:33 | Re: Parallel query and temp_file_limit |