Re: Adding CommandID to heap xlog records

On Thu, 8 Sept 2022 at 23:24, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com> writes:
> > Please find attached a patch that adds the CommandId of the inserting
> > transaction to heap (batch)insert, update and delete records. It is
> > based on the changes we made in the fork we maintain for Neon.
>
> This seems like a very significant cost increment with returns
> to only a minuscule number of users. We certainly cannot consider
> it unless you provide some evidence that that impression is wrong.

Attached a proposed set of patches to reduce overhead of the inital patch.

0001: moves the RMGR-specific xl_info flag bits into their own field
in the xlog header

This new field is allocated in the 2-byte alignment hole in the xlog
record header (of which now 1 byte is left). This change was discussed
by Andres (cc-ed) earlier in [0], and this is a partial implementation
of the suggestion. With the patch we could merge the xl_heap and
xl_heap2 redo managers, but that is not implemented and not a goal of
this patchset - we're only enabling the change, not providing it.

The main difference between this patch and the proposed change of [0]
is that this patch only provides a single 8-bit field for rmgr use,
for both flag bits and record types, as opposed to separate fields for
record type and flag bits.

The reason for including this patch is to get free bits to use in the
xlog header - all other addressable bits in the xlhdr are in use
already; and it is much more difficult to find usable bits in the heap
xlog structs. They exist, but processing would be much more of a pain
than what it is now.

0002: add new wal_level = remote
This implements the same concept as v1; but now makes CommandId
presence optional. This presence is now indicated by the all-new
XLOG_HEAP_WITH_CID bit. CommandId is included when wal_level is at
least set to the new 'remote' value. wal_level=logical by extension
also includes this commandId.

Performance numbers for this patch seem to indicate no significant
regression: Runs of pgbench (options: -s 50 -c 4 -j 4 -T 900 -v) have
shown no immediately significant regression with wal_level = replica
when compared to master @ cbe6dd17 (master: 978tps, patched: 985tps).
Results for wal_level = remote are slightly worse than with wal_level
= replica, but acceptable nonetheless (wal_level=remote: 964tps,
=replica: 985tps). Apart from wal_level being changed between runs, it
was an otherwise default postgres configuration with shared_buffers
set to 10GB.

Kind regards,

Matthias van de Meent

[0] https://www.postgresql.org/message-id/20220715173731.6t3km5cww3f5ztfq%40awork3.anarazel.de