Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
Cc: vignesh C <vignesh21(at)gmail(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions
Date: 2019-11-16 13:14:23
Message-ID: CAA4eK1L5u=L42CjAb44CrgH4c10-ew8Q=1SqqJ0QSVNr-a6XaQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Nov 7, 2019 at 5:13 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> Some notes before commit:
> --------------------------------------
> 1.
> Commit message need to be changed for the first patch
> -------------------------------------------------------------------------
> A.
> > The memory limit is defined by a new logical_decoding_work_mem GUC, so for example we can do this
>
> SET logical_decoding_work_mem = '128kB'
>
> > to trigger very aggressive streaming. The minimum value is 64kB.
>
> I think this patch doesn't contain streaming, so we either need to
> reword it or remove it.
>
> B.
> > The logical_decoding_work_mem may be set either in postgresql.conf, in which case it serves as the default for all publishers on that instance, or when creating the
> > subscription, using a work_mem paramemter in the WITH clause (specifies number of kilobytes).
>
> We need to reword this as we have decided to remove the setting from
> the subscription side as of now.
>
> 2. I think we can change the message level in UpdateSpillStats() to DEBUG2.
>

I have made these modifications and additionally ran pgindent.

> 4. I think we can combine both patches and commit as one patch, but it
> is okay to commit them separately as well.
>

I am not sure if this is a good idea, so still kept them as separate.

Tomas, do let me know if you want to commit these or if you have any
comments, otherwise, I will commit these on Tuesday (19-Nov)?

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Attachment Content-Type Size
0001-Add-logical_decoding_work_mem-to-limit-ReorderBuffer.nov16.patch application/octet-stream 21.8 KB
0002-Track-statistics-for-spilling-of-changes-from-Reorde.nov16.patch application/octet-stream 11.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Magnus Hagander 2019-11-16 13:29:58 Re: BUG #16079: Question Regarding the BUG #16064
Previous Message parveen mehta 2019-11-16 11:31:35 Postgres on IBM z/OS 2.2.0 and 2.3.0