Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

On Wed, Sep 30, 2020 at 12:16 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Mon, Sep 28, 2020 at 2:22 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> >
> > I don't have a patch for this idea but you can refer [1]
> > (v25-0002-Issue-individual-invalidations-with-wal_level-lo) to see
> > what I was trying to say about registering the invalidations in the
> > form of ReorderBufferChange. We have something along the lines of what
> > I described above in that patch but we have removed it because we need
> > all the invalidations to be accumulated to handle aborts and we don't
> > want two different mechanisms to store invalidations.
>
> In some of the intermediate version of the logical-decoding, we had
> collected in form of changes and later we changed it so that we can
> execute all the invalidation on abort. I just browsed old patch and
> fetch the changes to collect the invalidation in form of changes. I
> have rebased on the current head so that we collect both in form of
> changes as well as collection of the invalidation. So if we get
> individiaual invalidation we execute them and on abort we execute all
> the invalidation together.
>

Yeah, this is in-line with what I had in mind but please update the
comments in ReorderBufferAddInvalidations why we need to collect this
in both the forms. The comments are there but expand them a bit more.

And you might need to update the below comment as well:
typedef struct ReorderBufferTXN
{
..
/*
* List of ReorderBufferChange structs, including new Snapshots and new
* CommandIds
*/
dlist_head changes;

I have tried to think of a way to capture these changes only in one
form to serve our purpose but couldn't come up with any good ideas but
if you can think of some idea to achieve that I am all ears.

Keisuke-San, can you please test Dilip's patch to see if this has
fixed your problem and whether you see any other problem with this?

--
With Regards,
Amit Kapila.