Re: Need a builtin way to run all tests faster manner

On 2017-03-06 19:45:27 -0500, Tom Lane wrote:
> Stephen Frost <sfrost(at)snowman(dot)net> writes:
> > * Andres Freund (andres(at)anarazel(dot)de) wrote:
> >> I'm not quite sure what the best way to attack this is, but I think we
> >> need to do something.
>
> > I tend to agree with this, though I haven't got any great answers,
> > unfortunately.
>
> I don't want to reduce test coverage either. I think the most painless
> way to improve matters would just be to work harder on running tests in
> parallel. I think most devs these days do most of their work on 4- or
> 8-core machines, yet almost everything except the core regression tests
> is depressingly serial. I think we could likely get a 2x or better
> reduction in total runtime without too much work just by attacking that.

A lot more probably, based on my preliminary tests / my local test
script.

I'm just not quite sure what the best way is to make it easier to run
tests in parallel within the tree.

The best I can come up so far is a toplevel target that creates the temp
install, starts a cluster and then runs the 'installcheck-or-check'
target on all the subdirectories via recursion. Individual makefiles can
either use the pre-existing cluster (most of of contrib for example), or
use the temporary install and run their pre-existing check target using
that (the tap tests, test_decoding, ...).

Requires editing a bunch of Makefiles to take advantage. But I don't
really see anything that doesn't require that.

- Andres