Re: Testing "workers launched" in expected output? Really?

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> On Fri, Mar 2, 2018 at 2:28 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> ... now I am on the warpath. I have no idea whether or not the diff
>> here is significant --- maybe it is --- but I am desperately unhappy
>> that we have expected-output files that will fail if fewer than the
>> expected number of workers launched.

> Unless this is causing actual failures I don't think we should change
> it. It would be very sad if we started routinely getting Workers
> Launched < Workers Planned due to some newly-introduced bug and had no
> idea it was happening because we'd hidden it to avoid imaginary
> buildfarm failures.

My point is that just because it isn't falling over on
relatively-lightly-loaded buildfarm machines doesn't mean that it won't
fall over in other environments. You don't want packagers cursing your
name while trying to push out an emergency security fix because the
regression tests are unstable in their environment. Or worse, they might
adopt the expedient of skipping "make check" in their package recipes,
which is not a good idea from anyone's standpoint.

I should think that your recent experience with postgres_fdw (which is
evidently still broken, BTW, see rhinoceros) would have convinced you
that environmentally dependent regression test results are something to
studiously avoid. We have enough trouble with unforeseen test deltas;
putting in tests that have a blatantly obvious failure mechanism is
just taunting the software gods. Who tend to take their revenge in
unpleasant ways.

If you insist on actual failures, perhaps I'll go set up about four
concurrently-scheduled animals on prairiedog's host, and wait to see
what happens ...

regards, tom lane