Re: bogus: logical replication rows/cols combinations

From: Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>
To: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Pg Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: bogus: logical replication rows/cols combinations
Date: 2022-05-02 20:34:48
Message-ID: 47dd2cb9-4e96-169f-15ac-f9407fb54d43@enterprisedb.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 01.05.22 23:42, Tomas Vondra wrote:
> Imagine have a table with customers from different regions, and you want
> to replicate the data somewhere else, but for some reason you can only
> replicate details for one particular region, and subset of columns for
> everyone else. So you'd do something like this:
>
> CREATE PUBLICATION p1 FOR TABLE customers (... all columns ...)
> WHERE region = 'USA';
>
> CREATE PUBLICATION p1 FOR TABLE customers (... subset of columns ...)
> WHERE region != 'USA';
>
> I think ignoring the row filters and just merging the column lists makes
> no sense for this use case.

I'm thinking now the underlying problem is that we shouldn't combine
column lists at all. Examples like the above where you want to redact
values somehow are better addressed with something like triggers or an
actual "column filter" that works dynamically or some other mechanism.

The main purpose, in my mind, of column lists is if the tables
statically have different shapes on publisher and subscriber. Perhaps
for space reasons or regulatory reasons you don't want to replicate
everything. But then it doesn't make sense to combine column lists. If
you decide over here that the subscriber table has this shape and over
there that the subscriber table has that other shape, then the
combination of the two will be a table that has neither shape and so
will not work for anything.

I think in general we should be much more restrictive in how we combine
publications. Unless we are really sure it makes sense, we should
disallow it. Users can always make a new publication with different
settings and subscribe to that directly.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2022-05-02 20:54:39 Re: fix cost subqueryscan wrong parallel cost
Previous Message Tomas Vondra 2022-05-02 18:40:08 Re: bogus: logical replication rows/cols combinations