Re: [COMMITTERS] pgsql: Improved parallel make support

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> Very odd, but this completely blew up the first time I tried it.

> In file included from path.c:34:
> pg_config_paths.h:2:11: error: missing terminating " character

FWIW, I didn't replicate that, but I did get this during one attempt
with -j4:

/usr/bin/ranlib: archive member: libecpg.a(typename.o) size too large (archive \
member extends past the end of the file)
ar: internal ranlib command failed
make[5]: *** [libecpg.a] Error 1
make[5]: *** Deleting file `libecpg.a'
make[4]: *** [submake-ecpglib] Error 2
make[3]: *** [all-compatlib-recurse] Error 2
make[3]: *** Waiting for unfinished jobs....
/usr/bin/ranlib: can't stat file output file: libecpg.a (No such file or direct\
ory)
ar: internal ranlib command failed
make[4]: *** [libecpg.a] Error 1
make[3]: *** [all-ecpglib-recurse] Error 2
make[2]: *** [all-ecpg-recurse] Error 2
make[1]: *** [all-interfaces-recurse] Error 2
make[1]: *** Waiting for unfinished jobs....
In file included from gram.y:12101:
scan.c: In function 'yy_try_NUL_trans':
scan.c:16242: warning: unused variable 'yyg'
make: *** [all-src-recurse] Error 2

Examination of the stdout trace makes it appear that two independent
make runs were trying to build src/interfaces/ecpg/ecpglib/libecpg.a
concurrently. I haven't dug into it but I suspect that there are
multiple dependency chains leading to ecpg/ecpglib/. I wonder whether
what you saw was also the result of multiple recursion paths leading
to trying to build the same target at once. If so, that's going to
put a rather serious crimp in the idea of constraining build order
by adding more dependencies.

> On a subsequent retry, I got:
> ld: file not found: ../../../src/backend/postgres
> collect2: ld returned 1 exit status
> make[2]: *** [dict_snowball.so] Error 1

Yeah, I got that too, but adding all-backend/snowball-recurse to the
set of dependencies proposed in Peter's patch made it go away.
A cursory search for other appearances of -bundle_loader in the
make output suggests that contrib/ and src/test/regress/ are also
at risk. This leads me to the thought that concentrating knowledge
of this issue in src/Makefile is not the right way to go at it.
And, again, the more paths leading to a make attempt in the same
directory, the worse off we are as far as the first problem goes.
But surely the "make" guys recognized this risk and have a solution?
Otherwise parallel make would be pretty useless.

regards, tom lane