Re: Stopping logical replication protocol

On 10 May 2016 at 19:41, Vladimir Gordiychuk <folyga(at)gmail(dot)com> wrote:

>
> Fair enough. Though I don't understand why you'd be doing this often
>> enough that you'd care about reopening connections. What is the problem you
>> are trying to solve with this? The underlying reason you need this change?
>>
>
> First reason it clear API in pgjdc. Second reason it ability fast enough
> rollack to one of the previous LSN and repeat it. My use case for second
> reason - I have logical decoding extension that prepare only data as
> key-value pair without information about (insert, update, delete) for
> example if it delete as key I use primary key for table and as value null. Via
> pgjdc by replication protocol connects receiver data, consumer group
> changes to batch and send it to Kafka. If some problem occurs during
> delivery to kafka consumer, I should stop current replication, go back to
> success LSN and repeat all messages from success LSN. I think it operation
> can be quite common, but reopen connection or not stopping replication
> will increase latency.
>

It will, but not tons. The biggest cost (at least if there are any long
running xacts) will be replaying since the restart_lsn when you restart the
decoding session, which will happen whether you reconnect or just stop
decoding and return to command mode.

> Anyway, here's a draft patch that does the parts other than the reorder
>> buffer processing stop. It passes 'make check' and the src/test/recovery
>> tests, but I haven't written anything to verify the client-initiated abort
>> handling. You have test code for this and I'd be interested to see the
>> results.
>>
>
> What about keepAlive package when we already send/receive CopyDone? Is It
> really necessary?
>

No, I don't think it is, and I applied the change you made to suppress it.

> I think it's worth making the next step, where you allow reorder buffer
>> commit processing to be interrupted, into a separate patch on top of this
>> one. They're two separate changes IMO.
>>
>
> We will continue in the current thread, or new? I interesting in both
> patch for my solution and pgjbc driver.
>

Same thread, I just think these are two somewhat separate changes. One is
just in the walsender and allows return to command mode during waiting for
WAL. The other is more intrusive into the reorder buffer etc and allows
aborting decoding during commit processing. So two separate patches make
sense here IMO, one on top of the other.

I use git and just "git format-patch -2" to prepare a stack of two patches
from two separate commits.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services