Re: optimizing pg_upgrade's once-in-each-database steps

From: Nathan Bossart <nathandbossart(at)gmail(dot)com>
To: Ilya Gladyshev <ilya(dot)v(dot)gladyshev(at)gmail(dot)com>
Cc: Daniel Gustafsson <daniel(at)yesql(dot)se>, Robert Haas <robertmhaas(at)gmail(dot)com>, Jeff Davis <pgsql(at)j-davis(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: optimizing pg_upgrade's once-in-each-database steps
Date: 2024-08-06 19:20:14
Message-ID: ZrJ3bu4og_Uj_9qj@nathan
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sun, Aug 04, 2024 at 07:19:57PM +0100, Ilya Gladyshev wrote:
> -- End of review --

Thanks for the review. I've attempted to address all your feedback in v8
of the patch set. I think the names could still use some work, but I
wanted to get the main structure in place before trying to fix them.

> Regarding keeping the connections, the way I envisioned it entailed passing
> a list of connections from one check to the next one (or keeping a global
> state with connections?). I didn't concretely look at the code to verify
> this, so it's just an abstract idea.

My main concern with doing something like that is it could require
maintaining a huge number of connections when there are many databases.
GUCs like max_connections would need to be set accordingly. I'm a little
dubious that the gains would be enough to justify it.

--
nathan

Attachment Content-Type Size
v8-0001-introduce-framework-for-parallelizing-pg_upgrade-.patch text/plain 15.0 KB
v8-0002-use-new-pg_upgrade-async-API-for-subscription-sta.patch text/plain 8.9 KB
v8-0003-use-new-pg_upgrade-async-API-for-retrieving-relin.patch text/plain 13.1 KB
v8-0004-use-new-pg_upgrade-async-API-to-parallelize-getti.patch text/plain 3.2 KB
v8-0005-use-new-pg_upgrade-async-API-to-parallelize-repor.patch text/plain 3.3 KB
v8-0006-parallelize-data-type-checks-in-pg_upgrade.patch text/plain 12.3 KB
v8-0007-parallelize-isn-and-int8-passing-mismatch-check-i.patch text/plain 3.5 KB
v8-0008-parallelize-user-defined-postfix-ops-check-in-pg_.patch text/plain 5.2 KB
v8-0009-parallelize-incompatible-polymorphics-check-in-pg.patch text/plain 7.6 KB
v8-0010-parallelize-tables-with-oids-check-in-pg_upgrade.patch text/plain 3.4 KB
v8-0011-parallelize-user-defined-encoding-conversions-che.patch text/plain 4.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Noah Misch 2024-08-06 19:32:22 Re: Inval reliability, especially for inplace updates
Previous Message Heikki Linnakangas 2024-08-06 19:18:45 Re: Fix comments in instr_time.h and remove an unneeded cast to int64