Re: Cost of src/test/recovery and .../subscription tests

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> On Wed, Apr 19, 2017 at 1:28 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> I am going to say flat out that that's unacceptably long for
>> a test script that will be run dozens of times a day by the
>> buildfarm. There isn't any other test script that takes more
>> than circa 90 seconds on that machine, and I don't think this
>> one should either.

> I think that's bunk. If there are tests that are part of those test
> suites that are taking a long time to run and not providing meaningful
> coverage, then that's something that can be improved. However, I
> reject the argument that a test running for a long time is in itself
> bad. I'd rather have tests that run for a long time (and thus get run
> less often) than have no tests.

That's fine, the problem is that these are now in the buildfarm schedule,
meaning they are no longer in the "get run less often" group.

I've been poking at this further, and it seems like a lot of the problem
is not so much excessive coverage as badly implemented "wait" logic
causing the test to run a lot longer than it needs to. I already fixed
one such issue in 7834d20b5, and there seem to be more, both in the test
scaffolding and the code-under-test. Don't know how reasonable it is
to try to fix all this stuff at this point in the development cycle, but
I'm continuing to look into it.

regards, tom lane