Re: Allowing parallel pg_restore from pipe

From: Timothy Garnett <tgarnett(at)panjiva(dot)com>
To: sthomas(at)optionshouse(dot)com
Cc: Joachim Wieland <joe(at)mcknight(dot)de>, Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Dimitri Fontaine <dimitri(at)2ndquadrant(dot)fr>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Allowing parallel pg_restore from pipe
Date: 2013-05-16 17:16:17
Message-ID: CAPcyiQ0GZMgFJvbXG=DCVktj=MgcdDNSFoLMxNuRyYOa6S6e1g@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> If you need something like this short term, we actually found a way to do it
> ourselves for a migration we performed back in October. The secret is xargs
> with the -P option:
>
> xargs -I{} -P 8 -a table-list.txt \
> bash -c "pg_dump -Fc -t {} my_db | pg_restore -h remote -d my_db"
>
> Fill table-list.txt with as many, or as few tables as you want. The above
> example would give you 8 parallel threads. Well equipped systems may be able
> to increase this.
>
> Admittedly it's a gross hack, but it works. :)

I think you'd have to be real careful around foreign key constraints
for that to work.

Tim

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David E. Wheeler 2013-05-16 17:18:19 Re: Patch proposal: query result history in psql
Previous Message Tom Lane 2013-05-16 17:15:42 Re: Heap truncation without AccessExclusiveLock (9.4)