Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On 2020-04-18 11:10, Erik Rijkers wrote:
> On 2020-04-18 11:07, Erik Rijkers wrote:
>>>> Hi Erik,
>>>>
>>>> While setting up the cascading replication I have hit one issue on
>>>> base code[1]. After fixing that I have got one crash with streaming
>>>> on patch. I am not sure whether you are facing any of these 2
>>>> issues
>>>> or any other issue. If your issue is not any of these then plese
>>>> share the callstack and steps to reproduce.
>>
>> I figured out a few things about this. Attached is a bash script
>> test.sh, to reproduce:
>
> And the attached file, test.sh. (sorry)

It turns out I must have been mistaken somewhere. I probably missed
bugfix_in_schema_sent.patch)

I have just now rebuilt all the instances on top of master with these
patches:

> [v14-0001-Immediately-WAL-log-assignments.patch]
> [v14-0002-Issue-individual-invalidations-with.patch]
> [v14-0003-Extend-the-output-plugin-API-with-stream-methods.patch]
> [v14-0004-Gracefully-handle-concurrent-aborts-of-uncommitt.patch]
> [v14-0005-Implement-streaming-mode-in-ReorderBuffer.patch]
> [v14-0006-Add-support-for-streaming-to-built-in-replicatio.patch]
> [v14-0007-Track-statistics-for-streaming.patch]
> [v14-0008-Enable-streaming-for-all-subscription-TAP-tests.patch]
> [v14-0009-Add-TAP-test-for-streaming-vs.-DDL.patch]
> [v14-0010-Bugfix-handling-of-incomplete-toast-tuple.patch]
> [bugfix_in_schema_sent.patch]

(by the way: this build's regression tests 'ddl', 'toast', and
'spill' fail)

I seem now able to run all my test programs on these instances without
errors.

Sorry, I seem to have raised a false alarm (although there was initially
certainly a problem).

Erik Rijkers

>> There is a variable CRASH_IT that determines whether the whole thing
>> will fail (with a segmentation fault) or not. As attached it has
>> CRASH_IT=0 and does not crash. When you change that to CRASH_IT=1,
>> then it will crash. It turns out that this just depends on a short
>> wait state (3 seconds, on my machine) between setting up de
>> replication, and the running of pgbench. It's possible that on very
>> fast machines maybe it does not occur; we've had such difference
>> between hardware before. This is a i5-3330S.
>>
>> It deletes files so look it over before you run it. It may also
>> depend on some of my local set-up but I guess that should be easily
>> fixed.
>>
>> Can you let me know if you can reproduce the problem with this?
>>
>> thanks,
>>
>> Erik Rijkers
>>
>>
>>
>>>>
>>>> [1]
>>>> https://www.postgresql.org/message-id/CAFiTN-u64S5bUiPL1q5kwpHNd0hRnf1OE-bzxNiOs5zo84i51w%40mail.gmail.com
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Dilip Kumar
>>>> EnterpriseDB: http://www.enterprisedb.com