Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

From: Michail Nikolaev <michail(dot)nikolaev(at)gmail(dot)com>
To: Michael Paquier <michael(at)paquier(dot)xyz>
Cc: Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Andrey Borodin <amborodin86(at)gmail(dot)com>, Melanie Plageman <melanieplageman(at)gmail(dot)com>
Subject: Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements
Date: 2025-01-01 16:16:00
Message-ID: CANtu0oj6EXRq2US8gkz7ehELeECDsVAXq4R0iRfOaRW9GK=5Dw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello, everyone!

I’ve added several updates to the patch set:

* Automatic auxiliary index removal where applicable.
* Documentation updates to reflect recent changes.
* Optimization for STIR indexes: skipping datum setup, as they store only
TIDs.
* Numerous assertions to ensure that MyProc->xmin is invalid where
necessary.

I’d like to share some initial benchmark results (see attached graphs).
This involves building a B-tree index on (aid, abalance) in a pgbench setup
with scale 2000 (with WAL), while running a concurrent pgbench workload.

The patched version built the index in 68 seconds, compared to 117 seconds
with the master branch (mostly because of a single heap scan).
There appears to be no effect on the throughput of the concurrent pgbench.
The maximum snapshot age remains near zero.

I am going to continue to benchmark with different options: different HOT
setup, unique index, different index types and DB size (100+ GB).
If someone has some ideas about possible benchmark scenarios - please share.

Best regards,
Mikhail.

[image: image.png]

> [image: image.png]

Attachment Content-Type Size
v10-0010-Add-proper-handling-of-auxiliary-indexes-during-.patch application/octet-stream 28.7 KB
v10-0007-Improve-CREATE-REINDEX-INDEX-CONCURRENTLY-using-.patch application/octet-stream 101.5 KB
v10-0008-Concurrently-built-index-validation-uses-fresh-s.patch application/octet-stream 15.9 KB
v10-0009-Remove-PROC_IN_SAFE_IC-optimization.patch application/octet-stream 20.6 KB
v10-0011-Updates-index-insert-and-value-computation-logic.patch application/octet-stream 2.2 KB
v10-0006-Add-STIR-Short-Term-Index-Replacement-access-met.patch application/octet-stream 37.3 KB
v10-0003-Allow-advancing-xmin-during-non-unique-non-paral.patch application/octet-stream 43.2 KB
v10-0005-Allow-snapshot-resets-in-concurrent-unique-index.patch application/octet-stream 39.0 KB
v10-0004-Allow-snapshot-resets-during-parallel-concurrent.patch application/octet-stream 34.1 KB
v10-0001-This-is-https-commitfest.postgresql.org-50-5160-.patch application/octet-stream 17.5 KB
v10-0002-Add-stress-tests-for-concurrent-index-operations.patch application/octet-stream 8.0 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michail Nikolaev 2025-01-01 17:53:24 Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements
Previous Message Alexander Lakhin 2025-01-01 16:00:01 Re: Typos in the code and README