Lock-free compaction. Why not?

From: Ahmed Yarub Hani Al Nuaimi <ahmedyarubhani(at)gmail(dot)com>
To: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Lock-free compaction. Why not?
Date: 2024-07-09 04:58:02
Message-ID: CAF239vqdA5sOBr5j-AYhjvztzKwKBzUSGDpUo6QVBuEwcK_mOw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

So anyways I talked last week about lock-free vacuum. Since then I almost
finished creating a plugin that works on all Jetbrains products. It hooks
to the internal database tool and displays the internals of the database
and some stats. I would use this for my next phase: autovacuum with
compaction.
The thing is, after reading the code a million times, I still don't
understand why lock-free (or minimum locking) is such a big problem! Is it
that hard to lazily move tuples from one page to the other after
defragmenting it lazily?
I even made (a super simple equation) to calculate how much disk space can
be reclaimed, and created a full list of the advantages including
maintaining clustered indexes. This would of course be followed by
benchmarks.
Here is the wip plugin btw
https://github.com/ahmedyarub/database_internals_plugin/tree/2.0.0 which
should be released this week.
Like I don't even trust ChatGPT at all but I kept on trying to find reasons
for not doing it but I couldn't find any!
Such a trivial change that can bring a huge advantage. I just hope that
somebody could find me a reason for why it wouldn't work.

Really appreciate your patience,
Ahmed

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2024-07-09 05:05:37 Re: Slow catchup of 2PC (twophase) transactions on replica in LR
Previous Message Peter Smith 2024-07-09 04:22:46 Re: Pgoutput not capturing the generated columns