From: | Nathan Bossart <nathandbossart(at)gmail(dot)com> |
---|---|
To: | Ilya Gladyshev <ilya(dot)v(dot)gladyshev(at)gmail(dot)com> |
Cc: | Daniel Gustafsson <daniel(at)yesql(dot)se>, Robert Haas <robertmhaas(at)gmail(dot)com>, Jeff Davis <pgsql(at)j-davis(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: optimizing pg_upgrade's once-in-each-database steps |
Date: | 2024-08-01 17:44:35 |
Message-ID: | ZqvJg_lTTTt82BfQ@nathan |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Wed, Jul 31, 2024 at 10:55:33PM +0100, Ilya Gladyshev wrote:
> I like your idea of parallelizing these checks with async libpq API, thanks
> for working on it. The patch doesn't apply cleanly on master anymore, but
> I've rebased locally and taken it for a quick spin with a pg16 instance of
> 1000 empty databases. Didn't see any regressions with -j 1, there's some
> speedup with -j 8 (33 sec vs 8 sec for these checks).
Thanks for taking a look. I'm hoping to do a round of polishing before
posting a rebased patch set soon.
> One thing that I noticed that could be improved is we could start a new
> connection right away after having run all query callbacks for the current
> connection in process_slot, instead of just returning and establishing the
> new connection only on the next iteration of the loop in async_task_run
> after potentially sleeping on select.
Yeah, we could just recursively call process_slot() right after freeing the
slot. That'd at least allow us to avoid the spinning behavior as we run
out of databases to process, if nothing else.
> +1 to Jeff's suggestion that perhaps we could reuse connections, but perhaps
> that's a separate story.
When I skimmed through the various tasks, I didn't see a ton of
opportunities for further consolidation, or at least opportunities that
would help for upgrades from supported versions. The data type checks are
already consolidated, for example.
--
nathan
From | Date | Subject | |
---|---|---|---|
Next Message | Robert Haas | 2024-08-01 18:03:18 | Re: On disable_cost |
Previous Message | Matthias van de Meent | 2024-08-01 17:29:18 | Re: PG17beta2: SMGR: inconsistent type for nblocks |