Re: The Free Space Map: Problems and Opportunities

On Wed, Aug 18, 2021 at 7:45 AM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Tue, Aug 17, 2021 at 8:31 PM Peter Geoghegan <pg(at)bowt(dot)ie> wrote:
> > Lets say that VACUUM puts a non-zero usable amount of free space in
> > the FSM for a page that it marks all-visible/all-frozen at the same
> > time -- this is possible today, of course. To me this seems
> > contradictory, at least when there isn't much space -- which I believe
> > is common. It's at least a minor waste to set the VM bit.
>
> It seems to me that you are talking about the VM bit but making an
> argument that is more applicable to the FSM. If we think that the
> small amount of freespace in the page is going to result in the page
> being dirtied repeatedly as it cycles through a bunch of different
> tuples none of which stick around long term, then we might not want to
> advertise the freespace in the FSM, or we might want the FSM to
> disregard the fact that a small amount of freespace is present there.

I'm not quite sure if my argument applies to the VM or the FSM. It's a
bit tricky to talk about because I'm imagining a world in which the
concepts have shifted. And because the exact details of the data
structures are less clear (even if they were clear to me, the lack of
a shared vocabulary might confuse our discussion).

Maybe there isn't even a conventional FSM in this new world. Maybe
free space management works by focusing on recent events, and on
outcomes. This means that we store much less information about the
entire database, and much more information about very recent events.
In particular, I hope that free space management won't have to care
about how much free space most individual heap pages have. Perhaps
most heap pages are simply "closed", unavailable for reuse.

> However, if we do that, then we certainly want to also set the VM bit.
> Otherwise, we've got a page that we've effectively quiesced, by
> refusing to reuse the freespace, but the VM doesn't know that, so
> future index-only scans and VACUUM operations are penalized.

I agree that if we do that we'd want to also set the VM bit.

I'm imagining a world in which a "closed page" is synonymous with an
all-visible/all-frozen page. That doesn't mean that they're exactly
the same thing. For example, maybe a closed page is a page that is at
least *expected* to have its all-visible/all-frozen set by VACUUM the
next time it runs -- though more often it'll have been set already.

If you're looking at the system in real time with this improved
design, you'll see that pages that become closed will then also have
their VM bit set on a predictable timeline. Maybe we even set the VM
bit when some of the closed page's closed rows happen to be updated
(Freeze cutoff permitting) -- there isn't that much downside to VACUUM
betting that these "closed row updates" will be the last such updates
forever, or at least for a long time. This should at least be the case
once we add an optimized approach to freezing whole pages (pages whose
tuples start out unfrozen).

--
Peter Geoghegan