Re: Introduce XID age and inactive timeout based replication slot invalidation

On Mon, Mar 18, 2024 at 3:02 PM Bertrand Drouvot
<bertranddrouvot(dot)pg(at)gmail(dot)com> wrote:
>
> > > Hm. Are you suggesting inactive_timeout to be a slot level parameter
> > > similar to 'failover' property added recently by
> > > c393308b69d229b664391ac583b9e07418d411b6 and
> > > 73292404370c9900a96e2bebdc7144f7010339cf?
> >
> > Yeah, I have something like that in mind. You can prepare the patch
> > but it would be good if others involved in this thread can also share
> > their opinion.
>
> I think it makes sense to put the inactive_timeout granularity at the slot
> level (as the activity could vary a lot say between one slot linked to a
> subcription and one linked to some plugins). As far max_slot_xid_age I've the
> feeling that a new GUC is good enough.

Well, here I'm implementing the above idea. The attached v12 patches
majorly have the following changes:

1. inactive_timeout is now slot-level, that is, one can set it while
creating the slot either via SQL functions or via replication commands
or via subscription.
2. last_inactive_at and inactive_timeout are now tracked in on-disk
replication slot data structure.
3. last_inactive_at is now set even for non-walsenders whenever the
slot is released as opposed to initial versions of the patches setting
it only for walsenders.
4. slot's inactive_timeout parameter is now migrated to the new
cluster with pg_upgrade.
5. slot's inactive_timeout parameter is now synced to the standby when
failover is enabled for the slot.
6. Test cases are added to cover most of the above cases including new
invalidation mechanisms.

Following are some open points:

1. Where to do inactive_timeout invalidation exactly if not the checkpointer.
2. Where to do XID age invalidation exactly if not the checkpointer.
3. How to go about recomputing XID horizons based on max_slot_xid_age.
Does the slot's horizon's need to be adjusted in ComputeXidHorizons()?
4. New invalidation mechanisms interaction with slot sync feature.
5. Review comments on 0001 from Bertrand.

Please see the attached v12 patches.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com