Re: Inefficiency in parallel pg_restore with many tables

From: Nathan Bossart <nathandbossart(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Andrew Dunstan <andrew(at)dunslane(dot)net>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Inefficiency in parallel pg_restore with many tables
Date: 2023-09-13 22:47:53
Message-ID: 20230913224753.GA1237642@nathanxps13
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Sep 13, 2023 at 11:34:50AM -0700, Nathan Bossart wrote:
> On Sun, Sep 10, 2023 at 12:35:10PM -0400, Tom Lane wrote:
>> Other than those nitpicks, I like v6. I'll mark this RfC.
>
> Great. I've posted a v8 with your comments addressed in order to get one
> more round of cfbot coverage. Assuming those tests pass and there is no
> additional feedback, I'll plan on committing this in the next few days.

Upon closer inspection, I found a rather nasty problem. The qsort
comparator expects a TocEntry **, but the binaryheap comparator expects a
TocEntry *, and we simply pass the arguments through to the qsort
comparator. In v9, I added the requisite ampersands. I'm surprised this
worked at all. I'm planning to run some additional tests to make sure this
patch set works as expected.

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

Attachment Content-Type Size
v9-0001-Make-binaryheap-available-to-frontend-code.patch text/x-diff 8.8 KB
v9-0002-Add-function-for-removing-arbitrary-nodes-in-bina.patch text/x-diff 2.6 KB
v9-0003-Convert-pg_restore-s-ready_list-to-a-priority-que.patch text/x-diff 15.7 KB
v9-0004-Remove-open-coded-binary-heap-in-pg_dump_sort.c.patch text/x-diff 6.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Imseih (AWS), Sami 2023-09-13 23:09:19 Re: Jumble the CALL command in pg_stat_statements
Previous Message Euler Taveira 2023-09-13 21:38:08 Re: Support prepared statement invalidation when result types change