Re: pgsql: Fix fetching default toast value during decoding of in-progress

On Mon, Oct 7, 2024 at 7:03 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Mon, Oct 7, 2024 at 4:16 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Mon, Oct 7, 2024 at 4:14 PM Daniel Gustafsson <daniel(at)yesql(dot)se> wrote:
> > >
> > > > On 7 Oct 2024, at 12:22, Amit Kapila <akapila(at)postgresql(dot)org> wrote:
> > > >
> > > > Fix fetching default toast value during decoding of in-progress transactions.
> > >
> > > v14&v15 on adder seems a bit unhappy in the test_decoding test suite:
> > >
> >
> > oops, will check.
> >
>
> I think the reason for the instability of the test in 14 and 15 is
> that we don't have a predictable way to start streaming in those
> branches. We rely on the size of the reorder buffer. This can vary
> based on some background activity autoanalyze or others which can
> generate WAL records.
>

On further thinking, it appears to me that the reason is that the size
of records inserted may vary depending on the compression used for
toast (default_toast_compression=lz4) and or alignment. This can lead
to unpredictable behavior as to when to start streaming the changes,
leading to failure in some machines. I suggest changing the test to
use the PREPARE statement as that can reproduce the bug without the
fix as well. I'll test that change and commit it unless I see a
problem.

> So, will think some more on it tomorrow on how
> to make it reliable in those two branches. For other branches, we used
> GUC debug_logical_replication_streaming to control the streaming
> behavior.
>

It is better to keep the test the same in all branches so that any
future change is easier to backpatch and also for the sake of
consistency. So, even though the test works fine due to GUC
debug_logical_replication_streaming in branches >=16, I am planning to
change it to use the PREPARE statement.

--
With Regards,
Amit Kapila.