Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On 2020-04-16 11:46, Erik Rijkers wrote:
> On 2020-04-16 11:33, Dilip Kumar wrote:
>> On Tue, Apr 14, 2020 at 9:14 PM Erik Rijkers <er(at)xs4all(dot)nl> wrote:
>>>
>>> On 2020-04-14 12:10, Dilip Kumar wrote:
>>>
>>> > v14-0001-Immediately-WAL-log-assignments.patch +
>>> > v14-0002-Issue-individual-invalidations-with.patch +
>>> > v14-0003-Extend-the-output-plugin-API-with-stream-methods.patch+
>>> > v14-0004-Gracefully-handle-concurrent-aborts-of-uncommitt.patch+
>>> > v14-0005-Implement-streaming-mode-in-ReorderBuffer.patch +
>>> > v14-0006-Add-support-for-streaming-to-built-in-replicatio.patch+
>>> > v14-0007-Track-statistics-for-streaming.patch +
>>> > v14-0008-Enable-streaming-for-all-subscription-TAP-tests.patch +
>>> > v14-0009-Add-TAP-test-for-streaming-vs.-DDL.patch +
>>> > v14-0010-Bugfix-handling-of-incomplete-toast-tuple.patch
>>>
>>> applied on top of 8128b0c (a few hours ago)
>>
>
> I've added your new patch
>
> [bugfix_replica_identity_full_on_subscriber.patch]
>
> on top of all those above but the crash (apparently the same crash)
> that I had earlier still occurs (and pretty soon).
>
> server process (PID 1721) was terminated by signal 11: Segmentation
> fault
>
> I'll try to isolate it better and get a stacktrace
>
>
>> Hi Erik,
>>
>> While setting up the cascading replication I have hit one issue on
>> base code[1]. After fixing that I have got one crash with streaming
>> on patch. I am not sure whether you are facing any of these 2 issues
>> or any other issue. If your issue is not any of these then plese
>> share the callstack and steps to reproduce.

I figured out a few things about this. Attached is a bash script
test.sh, to reproduce:

There is a variable CRASH_IT that determines whether the whole thing
will fail (with a segmentation fault) or not. As attached it has
CRASH_IT=0 and does not crash. When you change that to CRASH_IT=1, then
it will crash. It turns out that this just depends on a short wait
state (3 seconds, on my machine) between setting up de replication, and
the running of pgbench. It's possible that on very fast machines maybe
it does not occur; we've had such difference between hardware before.
This is a i5-3330S.

It deletes files so look it over before you run it. It may also depend
on some of my local set-up but I guess that should be easily fixed.

Can you let me know if you can reproduce the problem with this?

thanks,

Erik Rijkers

>>
>> [1]
>> https://www.postgresql.org/message-id/CAFiTN-u64S5bUiPL1q5kwpHNd0hRnf1OE-bzxNiOs5zo84i51w%40mail.gmail.com
>>
>>
>> --
>> Regards,
>> Dilip Kumar
>> EnterpriseDB: http://www.enterprisedb.com