Re: In-placre persistance change of a relation

At Sun, 1 Sep 2024 22:15:00 +0300, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote in
> On 31/08/2024 19:09, Kyotaro Horiguchi wrote:
> This requires fsync()ing the per-xid undo log file every time a
> relation is created. I fear that can be a pretty big performance hit
> for workloads that repeatedly create and drop small tables. Especially
> if they're otherwise running with synchronous_commit=off. Instead of
> flushing the undo log file after every write, I'd suggest WAL-logging
> the undo log like regular relations and SLRUs. So before writing the
> entry to the undo log, WAL-log it. And with a little more effort, you
> could postpone creating the files altogether until a checkpoint
> happens, similar to how twophase state files are checkpointed
> nowadays.

After some delays, here’s the new version. In this update, UNDO logs
are WAL-logged and processed in memory under most conditions. During
checkpoints, they’re flushed to files, which are then read when a
specific XID’s UNDO log is accessed for the first time during
recovery.

The biggest changes are in patches 0001 through 0004 (equivalent to
the previous 0001-0002). After that, there aren’t any major
changes. Since this update involves removing some existing features,
I’ve split these parts into multiple smaller identity transformations
to make them clearer.

As for changes beyond that, the main one is lifting the previous
restriction on PREPARE for transactions after a persistence
change. This was made possible because, with the shift to in-memory
processing of UNDO logs, commit-time crash recovery detection is now
simpler. Additional changes include completely removing the
abort-handling portion from the pendingDeletes mechanism (0008-0010).

> I'd suggest using FullTransactionId. Doesn't matter much, but seems
> like a good future-proofing.

And, the patchset now uses full transaction ids.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center