"Missing" column in Postgres logical replication update message

From: Kevin Martin <martinkd(at)gmail(dot)com>
To: pgsql-general(at)lists(dot)postgresql(dot)org
Subject: "Missing" column in Postgres logical replication update message
Date: 2022-08-03 13:50:39
Message-ID: CAKYsA6U0KvvbF9Uq_eqXixaj7GB46cYB1b9VOfONSxmD6bMHMg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

We have a replication slot set up on a database in Postgres 12.8. For one
of the tables, when a row is created, we see a series of records come
through - an INSERT followed by a handful of UPDATEs. All of these messages
in the WAL files show all columns, except for the last UPDATE message,
which is missing one of the columns. (The column in question is a JSONB
field, and the data is not overly large - less than 1000 chars.) We
think this is causing the data to come into our data lake (via Stitch) with
that column as NULL.

My understanding is that all INSERT and UPDATE messages written to the
replication logs are supposed to include all columns. But I can't find a
definitive answer on that.

So, my first question is: Is it normal / expected for UPDATE messages in
the replication logs to exclude any columns in the table?

And, of course, if that is unexpected behavior, I'd love to hear any
thoughts on what may cause it.

Thanks.

-Kevin

FYI. I have this question posted also on StackOverflow:
https://stackoverflow.com/questions/73222107/missing-column-in-postgres-logical-replication-update-message

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Adrian Klaver 2022-08-03 14:47:52 Re: "Missing" column in Postgres logical replication update message
Previous Message Peter 2022-08-03 11:36:01 Why is my table continuousely written?