Re: Interval for launching the table sync worker

On 06/04/17 16:44, Masahiko Sawada wrote:
> On Thu, Apr 6, 2017 at 9:06 PM, Kyotaro HORIGUCHI
> <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote:
>> At Thu, 06 Apr 2017 17:02:14 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote in <20170406(dot)170214(dot)263553093(dot)horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
>>> At Thu, 6 Apr 2017 16:15:33 +0900, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote in <CAD21AoCrcwi3SwKKOW_efwW0finxyycLgsbw09n5uy2sxneO_A(at)mail(dot)gmail(dot)com>
>>>> On Thu, Apr 6, 2017 at 3:45 PM, Kyotaro HORIGUCHI
>>>> <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote:
>>>>> I was thinking the same.
>>>>>
>>>>> At Thu, 6 Apr 2017 11:33:22 +0900, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote in <CAD21AoDCnyRJDUY=ESVVe68AukvOP2dFomTeBFpAd1TiFbjsGg(at)mail(dot)gmail(dot)com>
>>>>>> Hi all,
>>>>>>
>>>>>> While testing table sync worker for logical replication I noticed that
>>>>>> if the table sync worker of logical replication failed to insert the
>>>>>> data for whatever reason, the table sync worker process exits with
>>>>>> error. And then the main apply worker launches the table sync worker
>>>>>> again soon without interval. This routine is executed at very high
>>>>>> frequency without interval.
>>>>>>
>>>>>> Should we do put a interval (wal_retrieve_interval or make a new GUC
>>>>>> parameter?) for launching the table sync worker?
>>
>> Hmm. I was playing with something wrong. Now I see the invervals
>> 5 seconds. No problem.
>
> Yeah, this issue happens only in the table sync worker.
>
>>
>>>>> After introducing encoding conversion, untranslatable characters
>>>>> in a published table causes this situation.
>>>>
>>>> I think it's better to make a new GUC parameter for the table sync
>>>> worker. Having multiple behaviors in wal_retrieve_retry_interval is
>>>> not good idea. Thought?
>>
>> So, this is working, sorry.
>>
>>> I prefer subscription option than GUC. Something like following.
>>>
>>> CREATE SUBSCRIPTION s1 CONNECTION 'blah'
>>> PUBLICATION p1 WITH (noreconnect = true);
>>>
>>> Stored in pg_subscription?

I don't think that's a very good solution, you'd lose replication on
every network glitch, upstream server restart, etc.

>>>
>>>>> Interval can reduce
>>>>> the frequence of reconnecting, but I think that walreciever
>>>>> should refrain from reconnecting on unrecoverable(or repeating)
>>>>> error in walsender.
>>>>>
>>>>
>>>> Yeah, that's also considerable issue.
>>>
>>> But not to do now.
>>
>
> I've added this as an open item, and sent a patch for this.
>

I am not exactly sure what's the open item from this thread. To use the
wal_retrieve_interval to limit table sync restarts?

--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services