Re: free space map and visibility map

I'd like to have a comment from Heikki or Tom.

At Mon, 27 Mar 2017 16:49:08 +0900, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote in <CAD21AoAnCw8y37dJSEhdbpue5H1v5FVLKUA_uh2MhZe331HNyw(at)mail(dot)gmail(dot)com>
> On Mon, Mar 27, 2017 at 2:38 PM, Kyotaro HORIGUCHI
> <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote:
> > Other than by FPI, FSM update is omitted when record LSN is older
> > than page LSN. If heap page is evicted but FSM page is not after
> > vacuuming and before power cut, replaying HEAP2_CLEAN skips
> > update of FSM even though FPI is not attached. Of course this
> > cannot occur on standby. One FSM page covers as many heap pages
> > as about 4k, so FSM can stay far longer than heap pages.
> >
> > ALL_FROZEN is set with other than HEAP2_FREEZE_PAGE. When a page
> > is already empty when entering lazy_sacn_heap, or a page of
> > non-indexed heap is empitied in lazy_scan_heap, HRAP2_VISIBLE is
> > issued to set ALL_FROZEN.
> >
> > Perhaps the problem will be fixed by forcing heap_xlog_visible to
> > update FSM (addition to FREEZE_PAGE), or the same in
> > heap_xlog_clean. (As menthined in the previous mail, I prefer the
> > latter.)
>
> Maybe it's enough just to make both heap_xlog_visible and
> heap_xlog_freeze_page forcibly updates the FSM (heap_xlog_freeze_page
> might be unnecessary). Because the problem happens on the page that is
> full according to FSM but is empty and marked as all-visible or

It would work and straightforward.

Currently FSM seems to be assumed as a part of heap from the view
of WAL. From the point of view, the problem is heap_xlog_clean
omits updating FSM for certain cases. My only concern is whether
updating heap information by visibility map record is right or
not. The code indents to reduce FSM updates without having
problem. For the insert/update cases, the problem is too-large
freespace information in FSM can cause needless fetches of heap
pages. But things are a bit different for the clean case. The
problem is too-small freespace information that causes
everlasting empty pages.

I dug out the original discussion. The mention on this was found
here.

https://www.postgresql.org/message-id/24334.1225205478%40sss.pgh.pa.us

Tom Lane wrote:
| Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> writes:
| > One issue with this patch is that it doesn't update the FSM at all when
| > pages are restored from full page images. It would require fetching the
| > page and checking the free space on it, or peeking into the size of the
| > backup block data, and I'm not sure if it's worth the extra code to do that.
|
| I'd vote not to bother, at least not in the first cut. As you say, 100%
| accuracy isn't required, and I think that in typical scenarios an
| insert/update that causes a page to become full would be relatively less
| likely to have a full-page image.

This is the 'first cut' shape, which hadn't cause a apparent
problem without ALL_FROZEN.

> all-frozen. Though heap_xlog_clean loads the heap page to the memory
> for redo operation, forcing heap_xlog_clean to update FSM might be
> overkill for this solution. Because it can happen on every pages that
> are not marked as neither all-visible nor all-frozen. Basically 100%

I'm not sure that it is defeinitely not an overkill but it seems
to me the same with the 20% rule of insert/update cases. We must
avoid 0% or too-small (under 20%?) FSM info on heap_clean for the
case especially for FREEZEing.

> accuracy of FSM is not required. On the other hand, if we makes

Yes, what is needed here is not accuracy, but miminum guratantee
not to cause a critical problem.

> heap_xlog_visible updates the FSM, it requires to load both heap page
> and FSM page, which can also be overhead. Another idea is, we can
> heap_xlog_visible to have the freespace of corresponding heap page,
> and then update FSM during recovery.

I haven't considered it. Counting freepsace by visiblilty logs is
worse in I/O perspective. Seems somewhat arbitrary but having
freespace in VM records seems to work.

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center