Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Tue, Nov 19, 2019 at 5:25 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Sat, Nov 16, 2019 at 6:44 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Thu, Nov 7, 2019 at 5:13 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > >
> > > Some notes before commit:
> > > --------------------------------------
> > > 1.
> > > Commit message need to be changed for the first patch
> > > -------------------------------------------------------------------------
> > > A.
> > > > The memory limit is defined by a new logical_decoding_work_mem GUC, so for example we can do this
> > >
> > > SET logical_decoding_work_mem = '128kB'
> > >
> > > > to trigger very aggressive streaming. The minimum value is 64kB.
> > >
> > > I think this patch doesn't contain streaming, so we either need to
> > > reword it or remove it.
> > >
> > > B.
> > > > The logical_decoding_work_mem may be set either in postgresql.conf, in which case it serves as the default for all publishers on that instance, or when creating the
> > > > subscription, using a work_mem paramemter in the WITH clause (specifies number of kilobytes).
> > >
> > > We need to reword this as we have decided to remove the setting from
> > > the subscription side as of now.
> > >
> > > 2. I think we can change the message level in UpdateSpillStats() to DEBUG2.
> > >
> >
> > I have made these modifications and additionally ran pgindent.
> >
> > > 4. I think we can combine both patches and commit as one patch, but it
> > > is okay to commit them separately as well.
> > >
> >
> > I am not sure if this is a good idea, so still kept them as separate.
> >
>
> I have committed the first patch. I will commit the second one
> related to stats of spilled xacts on Thursday. The second patch needs
> catalog version bump as well because we are modifying the catalog
> contents in that patch.
>

Committed the second one as well. Now, we can move to a review of
patches for "streaming of in-progress transactions".

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com