From: | Rory Campbell-Lange <rory(at)campbell-lange(dot)net> |
---|---|
To: | Robert Blayzor <rblayzor(dot)bulk(at)inoc(dot)net> |
Cc: | pgsql-performance(at)postgresql(dot)org |
Subject: | Re: PgSQL 14 - Logical Rep - Single table multiple publications? |
Date: | 2022-08-02 14:57:39 |
Message-ID: | Yuk7YwXqd2ETishS@campbell-lange.net |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-performance |
On 02/08/22, Robert Blayzor (rblayzor(dot)bulk(at)inoc(dot)net) wrote:
> Is it possible to have a single subscriber table contact multiple publishers
> and just insert all of the data into a single table on the subscriber? ie:
> merge type replication. There are no primary/FK constraints, etc. The
> records are just time based audit log type data...
Your use case meets, I think, the third "typical use case" listed at
https://www.postgresql.org/docs/current/logical-replication.html, namely
"Consolidating multiple databases into a single one (for example for
analytical purposes)."
I've just been testing aggregating all the data in one schema across 300
publisher databases into 5 subscriber schemas on two Postgresql 14 clusters on
the same machine. Each of 60 publisher tables are aggregating into a
single table on the subscriber.
Special care must be taken with the "replica identity" of published
tables, as set out at
https://www.postgresql.org/docs/current/logical-replication-publication.html.
For example, you may need a unique identifying column for each source
table in addition to the normal row identifier to differentiate *this*
table's id 1 row from the *other* table's id 1 row, otherwise the
subscriber won't be able to identify the row to delete if this table's
id 1 row is deleted (for example).
Although this seems to work fine with native replication, the pglogical
extension has more knobs. For instance, the
pglogical.wait_for_subscription_sync_complete function is useful to ensure that
sync finishes when part of a migration.
Rory
From | Date | Subject | |
---|---|---|---|
Next Message | bruno da silva | 2022-08-02 15:08:05 | Re: PostgresSQL 9.5.21 very slow to connect and perform basic queries |
Previous Message | Robert Blayzor | 2022-08-02 13:57:50 | PgSQL 14 - Logical Rep - Single table multiple publications? |