Re: Logical WAL streaming & START_REPLICATION

On 9 March 2018 at 04:49, Joe Hammerman <jhammerman(at)squarespace(dot)com> wrote:

> Although START_REPLICATION is called with the x_log position we desire,
> streaming is picked up from the current WAL log position.
>

That's how replication slots work. It's necessary to allow proper client-
and server-side crash recovery. They're really intended for use with a
replication origin or something similar client side to track client
progress.

> Is there a method to achieve our ends? Our alternative plan is to add a
> republish_count field to the the transaction, so that we can craft an SQL
> statement to update the rows from say, an hour ago to the present time.
>

What you'll need to do is find a way to delay sending replication slot
replay confirmations until you know the change is recorded persistently in
Kafka.

> that a flushed LSN may have been in the Kafka Broker Pool but not
consumed.

If your system will forget work on crash, it's not flushed, and you
shouldn't report it flushed.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services