Re: Logical replication existing data copy

On 25/02/17 00:05, Erik Rijkers wrote:
> On 2017-02-24 22:58, Petr Jelinek wrote:
>> On 23/02/17 01:41, Petr Jelinek wrote:
>>> On 23/02/17 01:02, Erik Rijkers wrote:
>>>> On 2017-02-22 18:13, Erik Rijkers wrote:
>>>>> On 2017-02-22 14:48, Erik Rijkers wrote:
>>>>>> On 2017-02-22 13:03, Petr Jelinek wrote:
>>>>>>
>>>>>>> 0001-Skip-unnecessary-snapshot-builds.patch
>>>>>>> 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch
>>>>>>> 0003-Fix-xl_running_xacts-usage-in-snapshot-builder.patch
>>>>>>> 0001-Use-asynchronous-connect-API-in-libpqwalreceiver.patch
>>>>>>> 0002-Fix-after-trigger-execution-in-logical-replication.patch
>>>>>>> 0003-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION.patch
>>>>>>> 0001-Logical-replication-support-for-initial-data-copy-v5.patch
>>>>>>
>>>>>> It works well now, or at least my particular test case seems now
>>>>>> solved.
>>>>>
>>>>> Cried victory too early, I'm afraid.
>>>>>
>>>>
>>>> I got into a 'hung' state while repeating pgbench_derail2.sh.
>>>>
>>>> Below is some state. I notice that master pg_stat_replication.syaye is
>>>> 'startup'.
>>>> Maybe I should only start the test after that state has changed. Any
>>>> of the
>>>> other possible values (catchup, streaming) wuold be OK, I would think.
>>>>
>>>
>>> I think that's known issue (see comment in tablesync.c about hanging
>>> forever). I think I may have fixed it locally.
>>>
>>> I will submit patch once I fixed the other snapshot issue (I managed to
>>> reproduce it as well, although very rarely so it's rather hard to test).
>>>
>>
>> Hi,
>>
>> Here it is. But check also the snapbuild related thread for updated
>> patches related to that (the issue you had with this not copying all
>> rows is yet another pre-existing Postgres bug).
>>
>
>
> The four earlier snapbuild patches apply cleanly, but
> then I get errors while applying
> 0001-Logical-replication-support-for-initial-data-copy-v6.patch:
>
>
> patching file src/test/regress/expected/sanity_check.out
> (Stripping trailing CRs from patch.)
> patching file src/test/regress/expected/subscription.out
> Hunk #2 FAILED at 25.
> 1 out of 2 hunks FAILED -- saving rejects to file
> src/test/regress/expected/subscription.out.rej
> (Stripping trailing CRs from patch.)
> patching file src/test/regress/sql/object_address.sql
> (Stripping trailing CRs from patch.)
> patching file src/test/regress/sql/subscription.sql
> (Stripping trailing CRs from patch.)
> patching file src/test/subscription/t/001_rep_changes.pl
> Hunk #9 succeeded at 175 with fuzz 2.
> Hunk #10 succeeded at 193 (offset -9 lines).
> (Stripping trailing CRs from patch.)
> patching file src/test/subscription/t/002_types.pl
> (Stripping trailing CRs from patch.)
> can't find file to patch at input line 4296
> Perhaps you used the wrong -p or --strip option?
> The text leading up to this was:
> --------------------------
> |diff --git a/src/test/subscription/t/003_constraints.pl
> b/src/test/subscription/t/003_constraints.pl
> |index 17d4565..9543b91 100644
> |--- a/src/test/subscription/t/003_constraints.pl
> |+++ b/src/test/subscription/t/003_constraints.pl
> --------------------------
> File to patch:
>
>
> Can you have a look?
>

Hi,

I think you are missing the other fixes (this specific error says that
you didn't apply
0002-Fix-after-trigger-execution-in-logical-replication.patch).

There is now a lot of fixes for existing code that this patch depends
on. Hopefully some of the fixes get committed soonish.

--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services