Re: Proposal to add page headers to SLRU pages

From: "Li, Yong" <yoli(at)ebay(dot)com>
To: Jeff Davis <pgsql(at)j-davis(dot)com>
Cc: Aleksander Alekseev <aleksander(at)timescale(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, "Bagga, Rishu" <bagrishu(at)amazon(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, "Debnath, Shawn" <sdn(at)ebay(dot)com>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, "Shyrabokau, Anton" <antons(at)ebay(dot)com>
Subject: Re: Proposal to add page headers to SLRU pages
Date: 2024-03-19 06:48:33
Message-ID: F9C587A3-E707-4C4E-9E83-124D963C85BA@ebay.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


>> - New comments have been added to pg_upgrade to mention the SLRU
>> page header change as the reason for upgrading clog files.
>
> That seems reasonable, but were any alternatives discussed? Do we have
> consensus that this is the right thing to do?

In general, there are two approaches. Either we convert the existing clog files,
or we don’t. The patch chooses to convert.

If we don’t, then the clog file code must be able to handle both formats. For,
XIDs in the range where the clog is written in the old format, segment and offset
computation must be done in one way, and for XIDs in a different range, it must
be computed in a different way. To avoid changing the format in the middle of a
page, which must not happen, the new format must start from a clean page,
possibly in a clean new segment. If the database is extremely small and has only
a few transactions on the first page of clog, then we must either convert the whole
page (effectively the whole clog file), or we must skip the rest of the XIDs on the
page and ask the database to start from XIDs on the second page on restart.
Also, we need to consider where to store the cut-off XID and when to remove it.
All these details feel very complex and error prone to me. Performing a one-time
conversion is the most efficient and straightforward approach to me.

>
> And if we use this approach, is there extra validation or testing that
> can be done?
>
> Regards,
> Jeff Davis

Unfortunately, the test requires a setup of two different versions of PG. I am not
aware of an existing test infrastructure which can run automated tests using two
PGs. I did manually verify the output of pg_upgrade.

Regards,
Yong

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2024-03-19 07:05:17 Re: speed up a logical replica setup
Previous Message Peter Eisentraut 2024-03-19 06:29:18 Re: What about Perl autodie?