Re: persist logical slots to disk during shutdown checkpoint

On Sun, Aug 20, 2023 at 8:40 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Sat, Aug 19, 2023 at 12:46 PM Julien Rouhaud <rjuju123(at)gmail(dot)com> wrote:
> >
> > On Sat, 19 Aug 2023, 14:16 Amit Kapila, <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >>
> >
> >> The idea discussed in the thread [1] is to always persist logical
> >> slots to disk during the shutdown checkpoint. I have extracted the
> >> patch to achieve the same from that thread and attached it here. This
> >> could lead to some overhead during shutdown (checkpoint) if there are
> >> many slots but it is probably a one-time work.
> >>
> >> I couldn't think of better ideas but another possibility is to mark
> >> the slot as dirty when we update the confirm_flush LSN (see
> >> LogicalConfirmReceivedLocation()). However, that would be a bigger
> >> overhead in the running server as it could be a frequent operation and
> >> could lead to more writes.
> >
> >
> > Yeah I didn't find any better option either at that time. I still think that forcing persistence on shutdown is the best compromise. If we tried to always mark the slot as dirty, we would be sure to add regular overhead but we would probably end up persisting the slot on disk on shutdown anyway most of the time, so I don't think it would be a good compromise.
> >
>
> The other possibility is that we introduce yet another dirty flag for
> slots, say dirty_for_shutdown_checkpoint which will be set when we
> update confirmed_flush LSN. The flag will be cleared each time we
> persist the slot but we won't persist if only this flag is set. We can
> then use it during the shutdown checkpoint to decide whether to
> persist the slot.

There are already two booleans controlling dirty-ness of slot, dirty
and just_dirty. Adding third will created more confusion.

Another idea is to record the confirm_flush_lsn at the time of
persisting the slot. We can use it in two different ways 1. to mark a
slot dirty and persist if the last confirm_flush_lsn when slot was
persisted was too far from the current confirm_flush_lsn of the slot.
2. at shutdown checkpoint, persist all the slots which have these two
confirm_flush_lsns different.

--
Best Wishes,
Ashutosh Bapat