Re: Challenges preventing us moving to 64 bit transaction id (XID)?

I'll be uploading three patches (in a subsequent reply to this thread, so I
can do it via email):

- 64-bit integer GUCs
- Using ClogPageNumber and CLOG_PAGENO_FMT in places where clog pages are
currently declared as int or printed as a %d
- Using XID_FMT in places where xid's are printed with format %u, and
XID_BITS for one or two cases when we need to variant the code based on 32
or 64 bit xids.

These changes don't affect field sizes at all yet. They compile, build, and
run check-world cleanly.

64-bit GUCs were used in the original patch set to handle several
autovacuum_* gucs that needed to become 64-bit. This seems like a useful
thing to commit independent of what we do with xids.

The ClogPageNumber patch gives a name to the kind of page index that is
returned by TransactionIdToPage. The intent is to improve code clarity for
now. In the original patch it was modified from int to uint64.

In the original patch set xids were formatted using XID_FMT, and XID_FMT was
defined as INT64_FORMAT. This patch changes all the same places to XID_FMT
but keeps the current format string as %u, so it just cleans things up and
makes the code less sensitive to the assumption that xids are 32 bits.

The big patch(s) after these first three patches reconstructs the logic from
the original patch set manually as well as I was able to manage. Those
changes have lots of TBDs and FIXMEs and don't necessarily compile, but
should at least apply cleanly for a while. There are also some residual
changes from the original patch set that I was unable to apply even manually
that will be factored out to a separate pseudo-patch.

A few observations about the big patch to come:

There appears to be widespread agreement that 64-bit xids should be
implemented by the table access method infrastructure, and that we need to
keep going in the direction of using FullTransactionId for the in-memory
representation. There is also widespread distaste for dealing with
multixacts as part of that. The original patch set came out a little bit
before table AMs were implemented, so redesigning it to support table AMs is
tbd. I'm not clear yet about how the CLOG was generalized for 64-bit xid's
except that ClogPageNumbers were based on a 64-bit xid and that the
autovacuum limit was increased to int64 and allowed to have a larger range.
In an excellent presentation that you can find from the FullTransactionId
wiki page, Thomas Munro suggested that the UNDO infrastructure might be used
as the mechanism to ensure that the effects of very old aborted transactions
in the CLOG would eventually be cleaned up.

The big patch(s) will be provided just so we don't entirely lose all the
work that went into creating the original patch set.

patches to follow in subsequent replies

-----
Jim Finnerty, AWS, Amazon Aurora PostgreSQL
--
Sent from: https://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html