GIN pending list cleanup during autoanalyze blocks cleanup by VACUUM

We generally only expect amvacuumcleanup() routines to be called
during VACUUM. But some ginvacuumcleanup() calls are an exception to
that general rule -- these are calls made during autoanalyze, where
ginvacuumcleanup() does real pending list cleanup work (not just a
no-op return). That'll only happen within an autoanalyze, and only
when no VACUUM took place before the ANALYZE. The high level goal is
to make sure that the av worker process won't neglect to call
ginvacuumcleanup() for pending list cleanup, even when there was no
VACUUM. This behavior was added when the GIN fastupdate/pending list
stuff was first introduced, in commit ff301d6e69.

The design of ANALYZE differs from the design of VACUUM in that only
ANALYZE will allocate an XID (typically in a call to
update_attstats()). ANALYZE can also hold an MVCC snapshot. This is
why ANALYZE holds back cleanup by VACUUM in another process, which
sometimes causes problems (say during pgbench) -- this much is fairly
well known. But there is also a pretty nasty interaction between this
aspect of ANALYZE, and the special GIN pending list cleanup path I
mentioned. This interaction makes the VACUUM-OldestXmin-held-back
situation far worse. The special analyze_only ginvacuumcleanup() calls
happen fairly late during the ANALYZE, during the window that ANALYZE
holds back OldestXmin values in other VACUUMs. This greatly increases
the extent of the problem, in the obvious way. GIN index pending list
cleanup will often take a great deal longer than the typical ANALYZE
tasks take -- it's a pretty resource intensive maintenance operation.
Especially if there are a couple of GIN indexes on the table.

This issue was brought to my attention by Nikolay Samokhvalov. He
reached out privately about it. He mentioned one problematic case
involving an ANALYZE lasting 45 minutes, or longer (per
log_autovacuum_min_duration output for the autoanalyze). That was
correlated with VACUUMs on other tables whose OldestXmin values were
all held back to the same old XID. I think that this issue ought to be
treated as a bug.

Jaime Casanova wrote a patch that does pending list cleanup using the
AV worker item infrastructure [1]. It's in the CF queue. Sounds like a
good idea to me. The goal of that patch is to take work out of the
insert path, when our gin_pending_list_limit-based limit is hit, but
offhand I imagine that the same approach could be used as a fix for
this issue, at least on HEAD.

[1] https://postgr.es/m/20210405063117.GA2478@ahch-to
--
Peter Geoghegan