Re: Further pg_upgrade analysis for many tables

On Wed, Nov 14, 2012 at 10:25:24AM -0500, Andrew Dunstan wrote:
>
> On 11/14/2012 10:08 AM, Bruce Momjian wrote:
> >On Wed, Nov 14, 2012 at 06:11:27AM +0200, Ants Aasma wrote:
> >>
> >>I agree that parallel restore for schemas is a hard problem. But I
> >>didn't mean parallelism within the restore, I meant that we could
> >>start both postmasters and pipe the output from dump directly to
> >>restore. This way the times for dumping and restoring can overlap.
> >Wow, that is a very creative idea. The current code doesn't do that,
> >but this has the potential of doubling pg_upgrade's speed, without
> >adding a lot of complexity. Here are the challenges of this approach:
> >
> >* I would need to log the output of pg_dumpall as it is passed to psql
> >so users can debug problems
>
>
> Instead of piping it directly, have pg_upgrade work as a tee,
> pumping bytes both to psql and a file. This doesn't seem terribly
> hard.

Right. It isn't hard.

> >* pg_upgrade never runs the old and new clusters at the same time for
> >fear that it will run out of resources, e.g. shared memory, or if they
> >are using the same port number. We can make this optional and force
> >different port numbers.
>
>
> Right.

OK.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +