Re: Introduce XID age and inactive timeout based replication slot invalidation

On Mon, Dec 30, 2024 at 11:05 AM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
>
> On Tue, Dec 24, 2024 at 10:37 PM Nisha Moond <nisha(dot)moond412(at)gmail(dot)com> wrote:
> >
> > On Fri, Dec 20, 2024 at 3:12 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >>
> >> On Mon, Dec 16, 2024 at 4:10 PM Nisha Moond <nisha(dot)moond412(at)gmail(dot)com> wrote:
> >> >
> >> > Here is the v56 patch set with the above comments incorporated.
> >> >
> >>
> >> Review comments:
> >> ===============
> >> 1.
> >> + {
> >> + {"idle_replication_slot_timeout", PGC_SIGHUP, REPLICATION_SENDING,
> >> + gettext_noop("Sets the duration a replication slot can remain idle before "
> >> + "it is invalidated."),
> >> + NULL,
> >> + GUC_UNIT_MS
> >> + },
> >> + &idle_replication_slot_timeout_ms,
> >>
> >> I think users are going to keep idele_slot timeout at least in hours.
> >> So, millisecond seems the wrong choice to me. I suggest to keep the
> >> units in minutes. I understand that writing a test would be
> >> challenging as spending a minute or more on one test is not advisable.
> >> But I don't see any test testing the other GUCs that are in minutes
> >> (wal_summary_keep_time and log_rotation_age). The default value should
> >> be one day.
> >
> >
> > +1
> > - Changed the GUC unit to "minute".
> >
> > Regarding the tests, we have two potential options:
> > 1) Introduce an additional "debug_xx" GUC parameter with units of seconds or milliseconds, only for testing purposes.
> > 2) Skip writing tests for this, similar to other GUCs with units in minutes.
> >
> > IMO, adding an additional GUC just for testing may not be worthwhile. It's reasonable to proceed without the test.
> >
> > Thoughts?
> >
> > The attached v57 patch-set addresses all the comments. I have kept the test case in the patch for now, it takes 2-3 minutes to complete.
> >
>
> Hi Nisha.
>
> I think we are often too quick to throw out perfectly good tests.
> Citing that some similar GUCs don't do testing as a reason to skip
> them just seems to me like an example of "two wrongs don't make a
> right".
>
> There is a third option.
>
> Keep the tests. Because they take excessive time to run, that simply
> means you should run them *conditionally* based on the PG_TEST_EXTRA
> environment variable so they don't impact the normal BF execution. The
> documentation [1] says this env var is for "resource intensive" tests
> -- AFAIK this is exactly the scenario we find ourselves in, so is
> exactly what this env var was meant for.
>
> Search other *.pl tests for PG_TEST_EXTRA to see some examples.
>

Thank you for the suggestion! I’ve added the tests under the
PG_TEST_EXTRA condition. Now, the '043_invalidate_inactive_slots.pl'
tests will only be executed when
PG_TEST_EXTRA=idle_replication_slot_timeout is set.

Attached the v58 patch set, addressing the above and the comments in
[1], [2], and [3].

[1] https://www.postgresql.org/message-id/CALDaNm14QrW5j6su%2BEAqjwnHbiwXJwO%2Byk73_%3D7yvc5TVY-43g%40mail.gmail.com
[2] https://www.postgresql.org/message-id/CAHut%2BPvDsM%3D%2BvTbM-xX6DD-PavONs2kGn03MZbCPGGL2t60TRA%40mail.gmail.com
[3] https://www.postgresql.org/message-id/CAHut%2BPs2ecNTfG3vsGb91CYpEzWtffyvkOzk1jqwhqTCwH8HQA%40mail.gmail.com

--
Thanks,
Nisha