Re: Assertion failure while streaming toasted data

On Tue, May 25, 2021 at 1:26 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:

>
>
> I have identified the cause of the issue, basically, the reason is if
> we are doing a multi insert operation we don't set the toast cleanup
> until we get the last tuple of the xl_multi_insert [1]. Now, with
> streaming, we can process the transaction in between the multi-insert
> but while doing that the "change->data.tp.clear_toast_afterwards" is
> set to false in all the tuples in this stream. And due to this we will
> not clean up the toast.
>

Thanks. That matches my understanding too.

>
> One simple fix could be that we can just clean the toast memory if we
> are processing in the streaming mode (as shown in the attached patch).
>

I am not entirely sure if it works correctly. I'd tried something similar,
but the downstream node using
my output plugin gets NULL values for the toast columns. It's a bit hard
to demonstrate that with the
test_decoding plugin, but if you have some other mechanism to test that
change with an actual downstream
node receiving and applying changes, it will be useful to test with that.

Thanks,
Pavan

--
Pavan Deolasee http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services