Re: POC: Cleaning up orphaned files using undo logs

On Mon, Aug 5, 2019 at 6:16 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> For zheap, we collect all the records of a page, apply them together
> and then write the entire page in WAL. The progress of transaction is
> updated at either transaction end (rollback complete) or after
> processing some threshold of undo records. So, generally, the WAL
> won't be for each undo record apply.

This explanation omits a crucial piece of the mechanism, because
Heikki is asking what keeps the undo from being applied multiple
times. When we apply the undo records to a page, we also adjust the
undo pointers in the page. Since we have an undo pointer per
transaction slot, and each transaction has its own slot, if we apply
all the undo for a transaction to a page, we can just clear the slot;
if we somehow end up back at the same point later, we'll know not to
apply the undo a second time because we'll see that there's no
transaction slot pointing to the undo we were thinking of applying. If
we roll back to a savepoint, or for some other reason choose to apply
only some of the undo to a page, we can set the undo record pointer
for the transaction back to the value it had before we generated any
newer undo. Then, we'll know that the newer undo doesn't need to be
applied but the older undo can be applied.

At least, I think that's how it's supposed to work. If you just
update the progress field, it doesn't guarantee anything, because in
the event of a crash, we could end up keeping the page changes but
losing the update to the progress, as they are part of separate undo
records.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company