Re: Add 64-bit XIDs into PostgreSQL 15

From: Maxim Orlov <orlovmg(at)gmail(dot)com>
To: Pavel Borisov <pashkin(dot)elfe(at)gmail(dot)com>
Cc: Justin Pryzby <pryzby(at)telsasoft(dot)com>, Aleksander Alekseev <aleksander(at)timescale(dot)com>, Postgres hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Stephen Frost <sfrost(at)snowman(dot)net>, Alexander Korotkov <aekorotkov(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Ilya Anfimov <ilan(at)tzirechnoy(dot)com>
Subject: Re: Add 64-bit XIDs into PostgreSQL 15
Date: 2022-07-06 12:55:02
Message-ID: CACG=eza0tv5H4mA+UF_Vk7_cPSQ6kFbzSxKFTuR1o=jmLoo9UQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi!

Here is a new version of the patchset with following changes:
- change unit test of page_conversion to address rare cfbot fails (the
reason of cfbot to time out was the unit test not written accurate enough);
- fix pg_upgarde on 32-bit systems;
- switch to XidList type (introduced by f10a025cfe97c1a34) for logical
replication, abandoning previously used Int64List type;
- use cat version instead of major version as a boundary from 32 to 64 bit
xids in pg_upgrade;
- this cat version boundary temporary set to 999999999 for pg_upgrade
testing purpose;
- also rebased to the actual master branch.

On 32-bit arch we have noticed pg_upgrade from 32 to 64 bit xids fails due
to different TOAST_MAX_CHUNK_SIZE. On a 64-bit xid page we have less
available space due to adding a heap page special. This leads to recalc of
TOAST_MAX_CHUNK_SIZE.

This was not a problem on 64 bit architectures, as padding bytes on 32-bit
xids on TOAST pages were enough to accommodate heap page special of 64-bit
xids with TOAST_MAX_CHUNK_SIZE unchanged. On 32 bits architectures padding
bytes were not enough and this needed TOAST_MAX_CHUNK_SIZE to be of
different size (on 64-bit xids version).

Changes of TOAST_MAX_CHUNK_SIZE lead to being unable to pg_upgrade onto
64-bit xids. This was a real problem, since TOAST of relation requires all
chunks to be the same size. In other words, we can not mix TOAST chunks of
previous (32-bit xid TOAST pages) with the new one with 64-bit xid TOAST
pages.

The solution was to use different specials for TOAST and heap pages. Since,
TOAST tuples can not have multixacts and does not need pd_multi_base on
page.

Thus, v39 is improved relative to v38 a lot.

As always, feel free to review and share your thoughts on a subject.

--
Best regards,
Maxim Orlov.

Attachment Content-Type Size
v39-0001-Use-64-bit-numbering-of-SLRU-pages.patch application/octet-stream 24.5 KB
v39-0005-Add-initdb-option-to-initialize-cluster-with-non.patch application/octet-stream 24.4 KB
v39-0002-Use-64-bit-format-to-output-XIDs.patch application/octet-stream 121.9 KB
v39-0004-Use-64-bit-pages-representation-in-SLRU-callers.patch application/octet-stream 23.7 KB
v39-0007-Use-64-bit-GUCs.patch application/octet-stream 25.8 KB
v39-0006-README.XID64.patch application/octet-stream 7.2 KB
v39-0003-Use-64-bit-FullTransactionId-instead-of-Epoch-xi.patch application/octet-stream 18.7 KB
v39-0008-Use-64-bit-XIDs.patch application/octet-stream 707.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Maxim Orlov 2022-07-06 13:00:04 Re: Custom tuplesorts for extensions
Previous Message Peter Eisentraut 2022-07-06 12:37:05 Re: doc: Fix description of how the default user name is chosen