Re: parallel pg_restore - WIP patch

Andrew Dunstan wrote:
>
>
> Andrew Dunstan wrote:
>>
>>
>>>
>>> this works better but there is something fishy still - using the same
>>> dump file I get a proper restore using pg_restore normally. If I
>>> however use -m for a parallel one I only get parts (in this case only
>>> 243 of the 709 tables) of the database restored ...
>>>
>>>
>>>
>>
>> Yes, there are several funny things going on, including some stuff
>> with dependencies. I'll have a new patch tomorrow with luck. Thanks
>> for testing.
>>
>>
>
> OK, in this version a whole heap of bugs are fixed, mainly those to do
> with dependencies and saved state. I get identical row counts in the
> source and destination now, quite reliably.

this looks much better (for a restore that usually takes 180min I can
get down to 72min using -m 4) - however especially with higher
concurrency I'm sometimes running into restore failures due to deadlocks
happening during constraint restoration (slightly redacted):

pg_restore: [archiver (db)] Error from TOC entry 7765; 2606 1460743180
FK CONSTRAINT fk_av_relations_av db_owner
pg_restore: [archiver (db)] could not execute query: ERROR: deadlock
detected
DETAIL: Process 18100 waits for AccessExclusiveLock on relation
1460818342 of database 1460815284; blocked by process 18103.
Process 18103 waits for AccessExclusiveLock on relation 1460818336 of
database 1460815284; blocked by process 18100.
HINT: See server log for query details.

ALTER TABLE ONLY foo
ADD CONSTRAINT fk_av_relations_av FOREIGN KEY (vs_id) REFERENCES
bar ...