It's a bit unfortunate that we're doing the pending list flush while the
vacuum memory is allocated at all. Is there any reason other than the way
the callbacks are defined that gin doesn't do the pending list flush before
vacuum does the heap scan and before it allocates any memory using
maintenance_work_mem?
(I'm guessing doing it after vacuum is finished would have different
problems with tuples in the pending queue not getting vacuumed?)