From: | Achilleas Mantzios <achill(at)matrix(dot)gatewaynet(dot)com> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | Re: JDBC: logical replication and LSN feedback |
Date: | 2017-09-19 14:14:16 |
Message-ID: | 254cb682-98e1-6966-48b7-1c931c5bd12e@matrix.gatewaynet.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general pgsql-jdbc |
On 19/09/2017 16:37, Yason TR wrote:
> Hi all,
>
> I am developing an application which connects to a logical replication slot, to consume the WAL events. These WAL events are then forwarded to a MQ broker.
>
> The heart of the code can be seen as:
>
> while (true) {
> Connection connection = null;
> PGReplicationStream stream = null;
>
> try {
> connection = DriverManager.getConnection("jdbc:postgresql://localhost:5432/db", properties);
> stream = connection.unwrap(PGConnection.class).getReplicationAPI().replicationStream().logical().withSlotName("slot").start();
>
> while (true) {
> final ByteBuffer buffer = stream.read();
>
> // ... MQ logic here ... omitted ...
>
> stream.setAppliedLSN(stream.getLastReceiveLSN());
> stream.setFlushedLSN(stream.getLastReceiveLSN());
> }
> } catch (final SQLException e) {
> // ... log exception ... omitted ...
> } finally {
> // ... close stream and connection ... omitted ...
> }
> }
>
> I notice some behavior which I cannot explain and would like to understand so I can alter my code:
>
> - When I restart the application, I notice that the application is retrieving the last event from the previous run again. The result is that this event is sent twice to the MQ broker after a restart of the application. Why is that? Isn't calling `setAppliedLSN(stream.getLastReceiveLSN())` and/or `setFlushedLSN(stream.getLastReceiveLSN())` enough to acknowledge an event, so it will removed from the WAL log and it will not be resent?
>
> - When receiving an event, the corresponding LSN from that event (which is sent in the payload) is not the same as the result of `stream.getLastReceivedLSN()`. Why is that? Which one should I use? Maybe this is correlated to my first question.
>
> - What is the difference between `setAppliedLSN(LSN)` and `setFlushedLSN(LSN)`? The Javadocs are not really helpful here.
The stages of a wal location generally go like : sent -> write -> flush -> replay , at least in terms of physical replication.
I guess applied=replayed ?
Note that from the docs : https://jdbc.postgresql.org/documentation/head/replication.html#logical-replication
it says :
"
In the event that replication has been restarted, it's will start from last successfully processed LSN that was sent via feedback to database.
"
>
> FYI, I also asked this question on https://stackoverflow.com/questions/46301578/postgres-jdbc-logical-replication-lsn-feedback.
>
> Thanks a lot and kind regards,
>
> Yason TR
>
>
--
Achilleas Mantzios
IT DEV Lead
IT DEPT
Dynacom Tankers Mgmt
From | Date | Subject | |
---|---|---|---|
Next Message | Yason TR | 2017-09-19 14:53:28 | Re: JDBC: logical replication and LSN feedback |
Previous Message | Job | 2017-09-19 13:47:16 | Insert large number of records |
From | Date | Subject | |
---|---|---|---|
Next Message | Yason TR | 2017-09-19 14:53:28 | Re: JDBC: logical replication and LSN feedback |
Previous Message | Yason TR | 2017-09-19 13:37:31 | JDBC: logical replication and LSN feedback |