Re: WIP: parallel GiST index builds

From: "Andrey M(dot) Borodin" <x4mmm(at)yandex-team(dot)ru>
To: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: WIP: parallel GiST index builds
Date: 2024-07-21 19:31:51
Message-ID: BC901E8E-8959-4FE7-A167-6F8E51C0D9D6@yandex-team.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Tomas!

> On 7 Jun 2024, at 20:41, Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> wrote:
>
> After looking into parallel builds for BRIN and GIN indexes, I was
> wondering if there's a way to do parallel builds for GiST too. I knew
> next to nothing about how GiST works, but I gave it a shot and here's
> what I have - the attached patch allows parallel GiST builds for the
> "unsorted" case (i.e. when the opclass does not include sortsupport),
> and does not support buffered builds.

I think this totally makes sense. I've took a look into tuples partitioning (for sorted build) in your Github and I see that it's complicated feature. So, probably, we can do it later.
I'm trying to review the patch as it is now. Currently I have some questions about code.

1. Do I get it right that is_parallel argument for gistGetFakeLSN() is only needed for assertion? And this assertion can be ensured just by inspecting code. Is it really necessary?
2. gistBuildParallelCallback() updates indtuplesSize, but it seems to be not used anywhere. AFAIK it's only needed to buffered build.
3. I think we need a test that reliably triggers parallel and serial builds.

As far as I know, there's a well known trick to build better GiST over PostGIS data: randomize input. I think parallel scan is just what is needed, it will shuffle tuples enough...

Thanks for working on this!

Best regards, Andrey Borodin.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alexander Lakhin 2024-07-21 20:08:08 Re: xid_wraparound tests intermittent failure.
Previous Message Andrew Dunstan 2024-07-21 18:46:28 Re: xid_wraparound tests intermittent failure.