Re: Allowing parallel pg_restore from pipe

From: Timothy Garnett <tgarnett(at)panjiva(dot)com>
To: Joachim Wieland <joe(at)mcknight(dot)de>
Cc: Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Dimitri Fontaine <dimitri(at)2ndquadrant(dot)fr>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Allowing parallel pg_restore from pipe
Date: 2013-04-25 20:01:38
Message-ID: CAPcyiQ1XGzp-sifGECagu3=gwjFSiwh-woVfyL85vfOF+zfNww@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Apr 24, 2013 at 5:47 PM, Joachim Wieland <joe(at)mcknight(dot)de> wrote:

> On Wed, Apr 24, 2013 at 4:05 PM, Stefan Kaltenbrunner <
> stefan(at)kaltenbrunner(dot)cc> wrote:
>
>> > What might make sense is something like pg_dump_restore which would have
>> > no intermediate storage at all, just pump the data etc from one source
>> > to another in parallel
>>
>
> That's right, I implemented that as an own output format and named it
> "migrator" I think, which wouldn't write each stream to a file as the
> directory output format does but that instead pumps it back into a restore
> client.
>
> I could revisit that patch for 9.4 if enough people are interested.
>
> Joachim
>

As the OP, I'll just note that my organization would definitely find use
for a parallel migrator tool as long as it supported doing a selection of
tables (i.e. -t / -T) in addition to the whole database and it supported or
we were able to patch in an option to cluster as part of the migration (the
equivalent of something like
https://github.com/tgarnett/postgres/commit/cc320a71 ).

Tim

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Shaun Thomas 2013-04-25 21:06:45 Re: Allowing parallel pg_restore from pipe
Previous Message Tom Lane 2013-04-25 19:54:55 Re: [ADMIN] Simultaneous index creates on different schemas cause deadlock?