Re: improving speed of make check-world

On 8/31/14 5:36 AM, Fabien COELHO wrote:
> Running "make -j2 check-world" does not work because "initdb" is not
> found by "pg_regress". but "make -j1 check-world" does work fine. It
> seems that some dependencies might be missing and there is a race
> condition between temporary install and running some checks?? Maybe it
> is not expected to work anyway? See below suggestions to make it work.

Here is an updated patch that fixes this problem.

The previous problem was simply a case were the make rules were not
parallel-safe.

For recursive make, we (through magic) set up targets like this:

check: check-subdir1-check check-subdir2-check

And with my old patch we added

check: temp-install

So the aggregate prerequisites were in effect something like

check: check-subdir1-check check-subdir2-check temp-install

And so there was nothing stopping a parallel make to descend into the
subdirectories before the temp install was set up.

What we need is additional prerequisites like

check-subdir1-check check-subdir2-check: temp-install

I have hacked this directly into the $(recurse) function, which is ugly.
This could possibly be improved somehow, but the effect would be the
same in any case.

With this, I can now run things like

make -C src/pl check -j3
make -C src/bin check -j8

A full make check-world -j$X is still prone to fail because some test
suites can't run in parallel with others, but that's a separate problem
to fix.

Note: We have in the meantime added logic to pg_regress to clean up the
temporary installation after the run. This changes that because
pg_regress is no longer responsible for the temporary installation.
pg_regress still cleans up the temporary data directory, so you still
get quite a bit of space savings. But the temporary installation is not
cleaned up. But since we would now only use a single temporary
installation, the disk space usage still stays in the same order of
magnitude.