Logical WAL streaming & START_REPLICATION

From: Joe Hammerman <jhammerman(at)squarespace(dot)com>
To: pgsql-jdbc(at)lists(dot)postgresql(dot)org
Subject: Logical WAL streaming & START_REPLICATION
Date: 2018-03-08 20:49:45
Message-ID: CAFZjsjCMLk4yQa8nhZ8Mj6fdSRs3gDws6G1YGdnD6LRGmi+DHg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin pgsql-jdbc

Hi pgsql-jdbc users email distribution list,

We are using the JDBC Driver bundled with the Debezium project. We are
trying to reset the replication stream from a WAL log point in the past.
This effort is in service of developing a disaster recovery plan, with the
idea that a flushed LSN may have been in the Kafka Broker Pool but not
consumed. The missing WAL files may still exist on disk, so it would be
ideal if we could begin streaming from a given WAL log file.

To that end, we have configured the Kafka Connect daemon to begin streaming
from a paritcular WAL log file.

Consider the following logs which show the creation of a new logical
replication slot:

https://pastebin.com/jMUms0Fs

Although START_REPLICATION is called with the x_log position we desire,
streaming is picked up from the current WAL log position.

Is there a method to achieve our ends? Our alternative plan is to add a
republish_count field to the the transaction, so that we can craft an SQL
statement to update the rows from say, an hour ago to the present time.

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Joe Hammerman 2018-03-08 21:00:02 Fwd: Logical WAL streaming & START_REPLICATION
Previous Message David G. Johnston 2018-03-08 20:20:38 Re: Trouble creating extensions: postgis and postgis_topology

Browse pgsql-jdbc by date

  From Date Subject
Next Message Joe Hammerman 2018-03-08 21:00:02 Fwd: Logical WAL streaming & START_REPLICATION
Previous Message Vladimir Sitnikov 2018-03-08 15:12:57 [pgjdbc/pgjdbc] af64ed: fix: allowEncodingChanges should allow set client_...