Re: Count and log pages set all-frozen by vacuum

On Wed, Nov 27, 2024 at 9:01 AM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
> On Tue, Nov 26, 2024 at 8:38 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> > > visibility map: 5 pages set all-visible, 4 pages set all-frozen.
> >
> > Which means that 5 pages were marked as only all-visible and 4 pages
> > were marked as all-frozen (and possibly all-visible too). The total
> > number of pages' whose VM status changed is the sum of these two
> > numbers, 9 pages. We would have no idea how many total VM bits were
> > set, though.
>
> To me, the above output means 9 bits changed, 5 of which were
> all-visible bits and 4 of which were all-frozen bits. It doesn't say
> whether they were the same pages or not (although we might be able to
> infer that based on only 3 of the 4 states being valid).
>
> If you want to count the number of pages that changed state, then I
> think the message wording needs to be different,

Agreed.

> but personally I
> think counting the number of flipped bits of each type seems easier to
> understand.

I agree that counting the number of flipped bits is easier to
understand. But I think there is still ambiguity when these two
numbers are mostly the same. For example, suppose that we report the
number of flipped bits and we have:

visibility map: 10 pages set all-visible, 10000 pages set all-frozen.

It's likely that many all-visible pages became all-frozen and 10
non-all-visible pages became all-visible. Overall, we can interpret it
that the number of all-frozen pages in the table increased much and
the number of all-visible pages (but not all-frozen) increased a bit
by this vacuum.

Then, suppose we have:

visibility map: 10000 pages set all-visible, 10 pages set all-frozen.

It's likely that many non-all-visible pages became all-visible but
most of which didn't become all-frozen. Overall, we can interpret it
that the number of all-visible pages (but not all-frozen) in the table
increased much and the number of all-frozen pages increased a bit by
this vacuum.

Finally, in case where we have:

visibility map: 10000 pages set all-visible, 10000 pages set all-frozen.

We can understand that 10000 pages newly became all-frozen, but have
no idea how many pages became all-visible but not all-frozen. It could
be even 0. Users might want to know it to understand how (auto)vacuum
and freezing are working well.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com