From: | Nathan Bossart <nathan(at)postgresql(dot)org> |
---|---|
To: | pgsql-committers(at)lists(dot)postgresql(dot)org |
Subject: | pgsql: Convert pg_restore's ready_list to a priority queue. |
Date: | 2023-09-19 21:32:46 |
Message-ID: | E1qiiKr-004pij-NS@gemulon.postgresql.org |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-committers |
Convert pg_restore's ready_list to a priority queue.
Presently, parallel restores spend a lot of time sorting this list
so that we pick the largest items first. With many tables, this
sorting can become a significant bottleneck. There are a couple of
reports from the field about this, and it is easy to reproduce.
This commit improves the performance of parallel pg_restore with
many tables by converting its ready_list to a priority queue, i.e.,
a binary heap. We will first try to run the highest priority item,
but if it cannot be chosen due to the lock heuristic, we'll do a
sequential scan through the heap nodes until we find one that is
runnable. This means that we might end up picking an item with a
much lower priority. However, we expect that we will typically be
able to pick one of the first few items, which should usually have
a relatively high priority.
Suggested-by: Tom Lane
Tested-by: Pierre Ducroquet
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/3612876.1689443232%40sss.pgh.pa.us
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/9bfd44bbde4261181bf94738f3b041c629c65a7e
Modified Files
--------------
src/bin/pg_dump/pg_backup_archiver.c | 198 ++++++++++-------------------------
1 file changed, 58 insertions(+), 140 deletions(-)
From | Date | Subject | |
---|---|---|---|
Next Message | Michael Paquier | 2023-09-20 00:32:38 | pgsql: psql: Reset query buffer of \e, \ef and \ev on error |
Previous Message | Heikki Linnakangas | 2023-09-19 16:30:04 | pgsql: Fix psql tab-completion for identifiers containing dollars. |