Re: Next step towards 64bit XIDs: Switch to FullTransactionId for PGPROC->xid and XLogRecord->xl_xid

Hi, Maxim, and Happy New Year!

On Mon, 1 Jan 2024 at 10:15, Maxim Orlov <orlovmg(at)gmail(dot)com> wrote:

> On Fri, 29 Dec 2023 at 16:36, Matthias van de Meent <
> boekewurm+postgres(at)gmail(dot)com> wrote:
>
>>
>> I don't think this is an actionable change, as this wastes 4 more bytes
>> (or 8 with alignment) in nearly all WAL records that don't use the
>> HEAP/HEAP2/XLOG rmgrs, which would then be up to 10 (if not 14, when
>> 64but-aligned) bytes per record. Unless something like [0] gets committed
>> this will add a significant write overhead to all operations, even if they
>> are not doing anything that needs an XID.
>>
>> Also, I don't think we need to support transactions that stay alive and
>> change things for longer than 2^31 concurrently created transactions, so we
>> could well add a fxid to each WAL segment header (and checkpoint record?)
>> and calculate the fxid of each record as a relative fxid off of that.
>>
>> Thank you for your reply. Yes, this is a good idea. Actually, this is
> exactly what I was trying to do at first.
> But in this case, we have to add more locks on TransamVariables->nextXid,
> and I've abandoned the idea.
> Maybe, I should look in this direction.
>
> On Sun, 31 Dec 2023 at 03:44, Michael Paquier <michael(at)paquier(dot)xyz> wrote:
>
>> And FWIW, echoing with Matthias, making these generic structures
>> arbitrary larger is a non-starter. We should try to make them
>> shorter.
>>
>
> Yeah, obviously, this is patch make WAL bigger. I'll try to look into the
> idea of fxid calculation, as mentioned above.
> It might in part be a "chicken and the egg" situation. It is very hard to
> split overall 64xid patch into smaller pieces
> with each part been a meaningful and beneficial for current 32xids
> Postgres.
>

The discussion quoted by you as OP [0] shows that there is a wide range of
opinions on whether we need 64-bit XIDs and what is the better way to
implement it. We can also continue discussing implementation details in
that thread [0].

I agree that the step towards something big should not make things worse
just now. Does increasing the WAL header for several bytes make performance
change detectable at all? Maybe it's worth simulating a workload with a
large amount of WAL records and seeing how it works. I think it's regarding
this patchset only and could remove or substantiate the main questions
about the current patchset.

[0]
https://www.postgresql.org/message-id/CACG=ezZe1NQSCnfHOr78AtAZxJZeCvxrts0ygrxYwe=pyyjVWA@mail.gmail.com

Kind regards,
Pavel Borisov.