Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

From: Michail Nikolaev <michail(dot)nikolaev(at)gmail(dot)com>
To: Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>
Cc: Michael Paquier <michael(at)paquier(dot)xyz>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Andrey Borodin <amborodin86(at)gmail(dot)com>, Melanie Plageman <melanieplageman(at)gmail(dot)com>
Subject: Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements
Date: 2025-01-30 01:00:00
Message-ID: CANtu0oi7d0_8oHpDPi_vFsuD0h71LNL4U2XXg0kq7iY_Ys3+SA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello, everyone!

> It was a wrong assumption. It looks like it is happening because of
prefetching. I'll try to add it in the validation phase.
This is an updated patch set, now prefetching is implemented.

Not validation works that way:
1) TIDs which are present in STIR auxiliary index but not present in target
index are loaded into tuplestore in sorted way
2) Then tuples from tuplestore are fetched one by one, but with underlying
prefetching of corresponding pages

Benchmark setups are the same as in [0].
Results show it works really well (see attachments).
I was unable to achieve consistent results for a few tests on the AWS (io2)
environment (and it was costly :) )

So, my next plan is:
1) wait a little bit for some comments from someone who still watches that
1-year going mainly solo thread :)
2) prepare a fresh new letter with patches, explanation, benchmark results
and so on.

Best regards,
Mikhail.

[0]:
https://www.postgresql.org/message-id/flat/CANtu0ojHAputNCH73TEYN_RUtjLGYsEyW1aSXmsXyvwf%3D3U4qQ%40mail.gmail.com#b18fb8efab086bc22af1cb015e187cb7

>

Attachment Content-Type Size
v13-0001-Add-stress-tests-for-concurrent-index-operations.patch application/octet-stream 8.0 KB
v13-0005-Add-STIR-Short-Term-Index-Replacement-access-met.patch application/octet-stream 37.0 KB
v13-0003-Allow-snapshot-resets-during-parallel-concurrent.patch application/octet-stream 34.1 KB
v13-0004-Allow-snapshot-resets-in-concurrent-unique-index.patch application/octet-stream 39.0 KB
v13-0002-Allow-advancing-xmin-during-non-unique-non-paral.patch application/octet-stream 43.6 KB
v13-0006-tuplestore-add-support-for-storing-Datum-values.patch application/octet-stream 17.3 KB
v13-0007-Improve-CREATE-REINDEX-INDEX-CONCURRENTLY-using-.patch application/octet-stream 110.0 KB
v13-0009-Remove-PROC_IN_SAFE_IC-optimization.patch application/octet-stream 20.6 KB
v13-0008-Concurrently-built-index-validation-uses-fresh-s.patch application/octet-stream 14.1 KB
v13-0010-Add-proper-handling-of-auxiliary-indexes-during-.patch application/octet-stream 28.7 KB
v13-0011-Updates-index-insert-and-value-computation-logic.patch application/octet-stream 2.2 KB
image/png 63.4 KB
image/png 64.4 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2025-01-30 03:10:54 Getting rid of CaseTestExpr (well, partially)
Previous Message Peter Smith 2025-01-30 00:51:13 Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.