Re: An idea for parallelizing COPY within one backend

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> Yeah, but it wouldn't take advantage of, say, the hack to disable WAL
> when the table was created in the same transaction.

In the context of a parallelizing pg_restore this would be fairly easy
to get around. We could probably teach the thing to combine table
creation and loading steps into one action (transaction) in most cases.
If that couldn't work because of some weird dependency chain, the
data loading transaction could be done as

BEGIN;
TRUNCATE table;
COPY table FROM stdin;
...
COMMIT;

which I believe already invokes the no-WAL optimization, and could
certainly be made to do so if not.

Obviously, pg_restore would have to be aware of whether or not it had
created that table in the current run (else it mustn't TRUNCATE),
but it would be tracking more or less exactly that info anyway to handle
dependency ordering.

regards, tom lane