Re: row filtering for logical replication

On Tue, Nov 30, 2021 at 11:37 AM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Tue, Nov 30, 2021 at 10:26 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Mon, Nov 29, 2021 at 8:40 PM Euler Taveira <euler(at)eulerto(dot)com> wrote:
> > >
> > > On Mon, Nov 29, 2021, at 7:11 AM, Amit Kapila wrote:
> > >
> > > I don't think it is a good idea to combine the row-filter from the
> > > publication that publishes just 'insert' with the row-filter that
> > > publishes 'updates'. We shouldn't apply the 'insert' filter for
> > > 'update' and similarly for publication operations. We can combine the
> > > filters when the published operations are the same. So, this means
> > > that we might need to cache multiple row-filters but I think that is
> > > better than having another restriction that publish operation 'insert'
> > > should also honor RI columns restriction.
> > >
> > > That's exactly what I meant to say but apparently I didn't explain in details.
> > > If a subscriber has multiple publications and a table is part of these
> > > publications with different row filters, it should check the publication action
> > > *before* including it in the row filter list. It means that an UPDATE operation
> > > cannot apply a row filter that is part of a publication that has only INSERT as
> > > an action. Having said that we cannot always combine multiple row filter
> > > expressions into one. Instead, it should cache individual row filter expression
> > > and apply the OR during the row filter execution (as I did in the initial
> > > patches before this caching stuff). The other idea is to have multiple caches
> > > for each action. The main disadvantage of this approach is to create 4x
> > > entries.
> > >
> > > I'm experimenting the first approach that stores multiple row filters and its
> > > publication action right now.
> > >
> >
> > We can try that way but I think we should still be able to combine in
> > many cases like where all the operations are specified for
> > publications having the table or maybe pubactions are same. So, we
> > should not give up on those cases. We can do this new logic only when
> > we find that pubactions are different and probably store them as
> > independent expressions and corresponding pubactions for it at the
> > current location in the v42* patch (in pgoutput_row_filter). It is
> > okay to combine them at a later stage during execution when we can't
> > do it at the time of forming cache entry.
>
> What about the initial table sync? during that, we are going to
> combine all the filters or we are going to apply only the insert
> filters?
>

AFAIK, currently, initial table sync doesn't respect publication
actions so it should combine all the filters. What do you think?

--
With Regards,
Amit Kapila.