Re: optimizing vacuum truncation scans

On Wed, Jul 8, 2015 at 9:46 PM, Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>
wrote:

> On Mon, Jun 29, 2015 at 3:54 PM, Jeff Janes <jeff(dot)janes(at)gmail(dot)com> wrote:
> >
> > Attached is a patch that implements the vm scan for truncation. It
> > introduces a variable to hold the last blkno which was skipped during the
> > forward portion. Any blocks after both this blkno and after the last
> > inspected nonempty page (which the code is already tracking) must have
> been
> > observed to be empty by the current vacuum. Any other process rendering
> the
> > page nonempty are required to clear the vm bit, and no other process can
> set
> > the bit again during the vacuum's lifetime. So if the bit is still set,
> the
> > page is still empty without needing to inspect it.
>
> The patch looks good and it improves the vacuum truncation speed
> significantly.
>
> > There is still the case of pages which had their visibility bit set by a
> > prior vacuum and then were not inspected by the current one. Once the
> > truncation scan runs into these pages, it falls back to the previous
> > behavior of reading block by block backwards. So there could still be
> > reason to optimize that fallback using forward-reading prefetch.
>
> The case, I didn't understand is that, how the current vacuum misses
> the page which
> was set by the prior vacuum?
>

The prior vacuum set them to all visible, but then doesn't delete them.
Either because it got interrupted or because there were still some pages
after them (at the time) that were not all empty.

The current vacuum skips them entirely on the forward scan because it
thinks it is waste of time to vacuum all visible pages. Since it skips
them, it doesn't know if they are empty as well as being all-visible.
There is no permanent indicator of the pages being all-empty, there is only
the inference based on the current vacuum's counters and protected by the
lock held on the table.

>
> The page should be counted either in skipped_pages or in
> nonempty_pages. Is it possible
> that a page doesn't comes under these two categories and it is not empty
> also?
>
> If the above doesn't exist, then we can directly truncate the relation
> from the highest block
> number of either nonempty_pages or skipped_pages to till the end of
> the relation.
>

Right, and that is what this does (provided the vm bit is still set, so it
does still have to loop over the vm to verify that it is still set, while
it holds the access exclusive lock).

The problem is that the pages between the two counters are not known to be
empty, and also not known to be nonempty. Someone has to be willing to go
check on those pages at some point, or else they will never be eligible for
truncation.

Cheers,

Jeff