Re: Parallel CREATE INDEX for GIN indexes

From: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Cc: Andy Fan <zhihuifan1213(at)163(dot)com>
Subject: Re: Parallel CREATE INDEX for GIN indexes
Date: 2024-06-24 00:58:16
Message-ID: 6db057fa-3990-4778-9578-aabc20f05db3@enterprisedb.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Here's a bit more cleaned up version, clarifying a lot of comments,
removing a bunch of obsolete comments, or comments speculating about
possible solutions, that sort of thing. I've also removed couple more
XXX comments, etc.

The main change however is that the sorting no longer relies on memcmp()
to compare the values. I did that because it was enough for the initial
WIP patches, and it worked till now - but the comments explained this
may not be a good idea if the data type allows the same value to have
multiple binary representations, or something like that.

I don't have a practical example to show an issue, but I guess if using
memcmp() was safe we'd be doing it in a bunch of places already, and
AFAIK we're not. And even if it happened to be OK, this is a probably
not the place where to start doing it.

So I've switched this to use the regular data-type comparisons, with
SortSupport etc. There's a bit more cleanup remaining and testing
needed, but I'm not aware of any bugs.

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Attachment Content-Type Size
v20240624-0001-Allow-parallel-create-for-GIN-indexes.patch text/x-patch 61.7 KB
v20240624-0002-Use-mergesort-in-the-leader-process.patch text/x-patch 12.6 KB
v20240624-0003-Remove-the-explicit-pg_qsort-in-workers.patch text/x-patch 10.2 KB
v20240624-0004-Compress-TID-lists-before-writing-tuples-t.patch text/x-patch 7.9 KB
v20240624-0005-Collect-and-print-compression-stats.patch text/x-patch 5.7 KB
v20240624-0006-Enforce-memory-limit-when-combining-tuples.patch text/x-patch 14.0 KB
v20240624-0007-Detect-wrap-around-in-parallel-callback.patch text/x-patch 8.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Ranier Vilela 2024-06-24 01:05:42 Re: Avoid incomplete copy string (src/backend/access/transam/xlog.c)
Previous Message Gurjeet Singh 2024-06-24 00:57:00 Proposal: Division operator for (interval / interval => double precision)