Re: Hot standby and removing VACUUM FULL

On Sat, 2009-11-21 at 20:20 +0200, Heikki Linnakangas wrote:

> VACUUM FULL does a peculiar hack: once it's done moving tuples, and
> before it truncates the relation, it calls RecordTransactionCommit to
> mark the transaction as committed in clog and WAL, but the transaction
> is still kept open in proc array. After it's done with truncating and
> other cleanup, normal transaction commit performs
> RecordTransactionCommit again as part of normal commit processing.
>
> That causes some headaches for Hot Standby, ie. it needs to know to not
> release locks yet when it sees the first commit record. At the moment,
> it simply ignores the first commit record, but that's very dangerous. If
> failover happens after the standby has seen the truncation record, but
> not the 2nd commit record for the transaction, all data moved from the
> truncated part will lost.

...

> So I guess what I'm asking is: Does anyone see any show-stoppers in
> removing VACUUM FULL, and does anyone want to step up to the plate and
> promise to do it before release?

I've just reviewed the VACUUM FULL patch to see if it does all we need
it to do, i.e. does the removal of HS code match the new VF patch.

My answer is it doesn't, we will still have the problem noted above for
catalog tables. So we still have a must-fix issue for HS, though that is
no barrier to the new VF patch.

Requirement is that

* we ignore first commit, since it has an identical xid to second
commit, so requires a special case to avoid breaking other checks

* we musn't truncate until we are certain transaction completes,
otherwise we will have a data loss situation (isolated to this
particular case only)

Proposal:

* (In normal running) Just before we issue the first VFI commit we send
a new type of WAL message to indicate VFI-in-progress, including the xid

* (In HS recovery) When we see first commit record for the VF xid we
ignore it, as we did in the original patch

* (In HS recovery) When we see relation truncate for the xid we ignore
it for now, but defer until after the second commit is processed

It ain't that great, but it means all of the "hack" is isolated to
specific HS-only code, which can be turned off if required.

--
Simon Riggs www.2ndQuadrant.com