Re: optimizing pg_upgrade's once-in-each-database steps

From: Nathan Bossart <nathandbossart(at)gmail(dot)com>
To: Daniel Gustafsson <daniel(at)yesql(dot)se>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Jeff Davis <pgsql(at)j-davis(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: optimizing pg_upgrade's once-in-each-database steps
Date: 2024-07-22 20:07:10
Message-ID: Zp677slGLwhPhVWy@nathan
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Jul 19, 2024 at 04:21:37PM -0500, Nathan Bossart wrote:
> However, while looking into this, I noticed that only one get_query
> callback (get_db_subscription_count()) actually customizes the generated
> query using information in the provided DbInfo. AFAICT we can do this
> particular step without running a query in each database, as I mentioned
> elsewhere [0]. That should speed things up a bit and allow us to simplify
> the AsyncTask code.
>
> With that, if we are willing to assume that a given get_query callback will
> generate the same string for all databases (and I think we should), we can
> run the callback once and save the string in the step for dispatch_query()
> to use. This would look more like what you suggested in the quoted text.

Here is a new patch set. I've included the latest revision of the patch to
fix get_db_subscription_count() from the other thread [0] as 0001 since I
expect that to be committed soon. I've also moved the patch that moves the
"live_check" variable to "user_opts" to 0002 since I plan on committing
that sooner than later, too. Otherwise, I've tried to address all feedback
provided thus far.

[0] https://commitfest.postgresql.org/49/5135/

--
nathan

Attachment Content-Type Size
v5-0001-pg_upgrade-retrieve-subscription-count-more-effic.patch text/plain 5.5 KB
v5-0002-move-live_check-variable-to-user_opts.patch text/plain 11.8 KB
v5-0003-introduce-framework-for-parallelizing-pg_upgrade-.patch text/plain 10.0 KB
v5-0004-use-new-pg_upgrade-async-API-for-subscription-sta.patch text/plain 9.0 KB
v5-0005-use-new-pg_upgrade-async-API-for-retrieving-relin.patch text/plain 12.9 KB
v5-0006-use-new-pg_upgrade-async-API-to-parallelize-getti.patch text/plain 3.3 KB
v5-0007-use-new-pg_upgrade-async-API-to-parallelize-repor.patch text/plain 3.4 KB
v5-0008-parallelize-data-type-checks-in-pg_upgrade.patch text/plain 12.4 KB
v5-0009-parallelize-isn-and-int8-passing-mismatch-check-i.patch text/plain 3.7 KB
v5-0010-parallelize-user-defined-postfix-ops-check-in-pg_.patch text/plain 5.3 KB
v5-0011-parallelize-incompatible-polymorphics-check-in-pg.patch text/plain 8.7 KB
v5-0012-parallelize-tables-with-oids-check-in-pg_upgrade.patch text/plain 3.5 KB
v5-0013-parallelize-user-defined-encoding-conversions-che.patch text/plain 4.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David Christensen 2024-07-22 20:55:20 [PATCH] GROUP BY ALL
Previous Message Andrew Dunstan 2024-07-22 19:53:08 Re: xid_wraparound tests intermittent failure.