Re: Transaction Snapshots and Hot Standby

Heikki Linnakangas wrote:
> BTW, we haven't talked about how to acquire a snapshot in the slave.
> You'll somehow need to know which transactions have not yet
> committed, but will in the future. In the master, we keep track of
> in-progress transaction in the ProcArray, so I suppose we'll need to
> do the same in the slave. Very similar to prepared transactions,
> actually. I believe the Abort records, which are not actually needed
> for normal operation, become critical here. The slave will need to
> put an entry to ProcArray for any new XLogRecord.xl_xid it sees in
> the WAL, and remove the entry at a Commit and Abort record. And clear
> them all at a shutdown record.

For reference, here is how I solved the snapshot problem in my
Summer-of-Code project last year, which dealt exactly with executing
read-only queries on PITR slaves (But sadly never came out of alpha
stage due to both my and Simon's lack of time)

The main idea was to invert the meaning of the xid array in the snapshot
struct - instead of storing all the xid's between xmin and xmax that are
to be considering "in-progress", the array contained all the xid's >
xmin that are to be considered "completed".

The current read-only snapshot (which "current" meaning the
corresponding state on the master at the time the last replayed wal
record was generated) was maintained in shared memory. It' xmin field
was continually updated with the (newly added) XLogRecord.xl_xmin
field, which contained the xid of the oldest running query on the
master, with a pruning step after each ReadOnlySnapshot.xmin update to
remove all entries < xmin from the xid array. If a commit was seen for
an xid, that xid was added to the ReadOnlySnapshot.xid array.

The advantage of this concept is that it handles snapshotting on the
slave without too much additional work for the master (The only change
is the addition of the xl_xmin field to XLogRecord). It especially
removes that need to track ShmemVariableCache->nextXid.

The downside is that the size of the read-only snapshot is theoretically
unbounded, which poses a bit of a problem if it's supposed to live
inside shared memory...

regards, Florian Pflug