Re: xid wraparound danger due to INDEX_CLEANUP false

On Sat, 18 Apr 2020 at 03:22, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
> On Thu, Apr 16, 2020 at 6:49 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> > Yea. _bt_vacuum_needs_cleanup() needs to check if
> > metad->btm_oldest_btpo_xact is older than the FreezeLimit computed by
> > vacuum_set_xid_limits() and vacuum the index if so even if INDEX_CLEANUP
> > false.
>
> I'm still fairly unclear on what the actual problem is here, and on
> how we propose to fix it. It seems to me that we probably don't have a
> problem in the case where we don't advance relfrozenxid or relminmxid,
> because in that case there's not much difference between the behavior
> created by this patch and a case where we just error out due to an
> interrupt or something before reaching the index cleanup stage. I
> think that the problem is that in the case where we do relfrozenxid,
> we might advance it past some XID value stored in the index metadata.
> Is that right?

I think advancing relfrozenxid past oldest_btpo_xact actually cannot
be a problem. If a subsequent vacuum sees oldest_btpo_xact is an old
xid, we can recycle pages. Before introducing to INDEX_CLEANUP =
false, we used to invoke either bulkdelete or vaucumcleanup at least
once in each vacuum. And thanks to relfrozenxid, a table is
periodically vacuumed by an anti-wraparound vacuum. But with this
feature, we can unconditionally skip both bulkdelete and
vacuumcleanup. So IIUC the problem is that since we skip both,
oldst_btpo_xact could be seen as a "future" xid during vacuum. Which
will be a cause of that vacuum misses pages which can actually be
recycled.

I think we can fix this issue by calling vacuumcleanup callback when
an anti-wraparound vacuum even if INDEX_CLEANUP is false. That way we can
let index AM make decisions whether doing cleanup index at least once
until XID wraparound, same as before. Originally the motivation of
disabling INDEX_CLEANUP was to skip index full scan when
anti-wraparound vacuum to reduce the execution time. By this
change, we will end up doing an index full scan also in some
anti-wraparound vacuum case but we still can skip that if there is no
recyclable page in an index.

Regards,

--
Masahiko Sawada http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services