Adding CommandID to heap xlog records

Hi,

The current WAL records generated by the Heap tableAM do not contain
the command ID of the query that inserted/updated/deleted the records.
The CID is not included in XLog because it is useful only to
visibility checks in an active read/write transaction, which currently
only appear in a primary node.

In Neon [0], we're using XLog to reconstruct page state, as opposed to
writing out dirty pages. This has the benefit of saving write IO for
dirty page writes, but this does mean that we need the CID in heap
insert/update/delete records to correctly mark the tuples, such that
modified pages that are flushed from the buffer pool get reconstructed
correctly. A more detailed write-up why we do this is here [1].

Neon does not need to be the only user of this API, as adding CID to
xlog records also allows the primary to offload (partial) queries to a
remote physical replica that would utilise the same transaction and
snapshot of the primary.
Right now, it's not possible to offload the read-component of RW
queries to a secondary [2]. The attached patch would make multi-node
transactions possible on systems with a single primary node and
multiple read replicas, without the need for prepared commits and
special extra code to achieve snapshot consistency, as a consistent
snapshot could be copied and used by physical replicas (think parallel
workers, but on a different server).

Please find attached a patch that adds the CommandId of the inserting
transaction to heap (batch)insert, update and delete records. It is
based on the changes we made in the fork we maintain for Neon.

Kind regards,

Matthias van de Meent

[0] https://github.com/neondatabase/neon/#neon
[1] https://github.com/neondatabase/neon/blob/main/docs/core_changes.md#add-t_cid-to-heap-wal-records
[2] At least not without blocking XLog replay of the primary
transaction on the secondary, due to the same issues that Neon
encountered: you need the CommandID to distinguish between this
transactions' updates in the current command and previous commands.