From: | Dilip Kumar <dilipbalaut(at)gmail(dot)com> |
---|---|
To: | Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> |
Cc: | Erik Rijkers <er(at)xs4all(dot)nl>, Kuntal Ghosh <kuntalghosh(dot)2007(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions |
Date: | 2020-06-12 06:08:23 |
Message-ID: | CAFiTN-uZRevjyjGY_v3OKbxFPz2zP9O6NhVK_BCVq94wgTRePA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Wed, Jun 10, 2020 at 5:11 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Wed, Jun 10, 2020 at 5:02 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> >
> > On Wed, Jun 10, 2020 at 4:00 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > >
> > > > 2. Files should not be closed at the end of the transaction:
> > > > Currently, files opened with BufFileCreateShared/BufFileOpenShared are
> > > > registered to be closed on EOXACT. Basically, we need to open the
> > > > changes file on the stream start and keep it open until stream stop,
> > > > so we can not afford to get it closed on the EOXACT. I have added a
> > > > flag for the same.
> > > >
> > >
> > > But where do we end the transaction before the stream stop which can
> > > lead to closure of this file?
> >
> > Currently, I am keeping the transaction only while creating/opening
> > the files and closing immediately after that, maybe we can keep the
> > transaction until stream stop, then we can avoid this changes, and we
> > can also avoid creating extra resource owner? What is your thought on
> > this?
> >
>
> I would prefer to keep the transaction until the stream stop unless
> there are good reasons for not doing so.
I am ready with the first patch set which replaces the temp file usage
in the worker with the buffile usage. (patch v27-0013 and v27-0014)
Open item:
- As of now, I have kept the buffile changes and the worker using
buffile as separate patches for review. Later I will make buffile
changes patch as a base patch and I will merge the worker changes with
the 0008 patch.
- Currently, while reading/writing the streaming/subxact files we are
reporting the wait event for example
'pgstat_report_wait_start(WAIT_EVENT_LOGICAL_SUBXACT_WRITE);', but
BufFileWrite/BufFileRead internally reports the read/write wait event.
So I think we can avoid reporting that? Basically, this part is still
I have to work upon, once we get the consensus then I can remove those
extra wait event from the patch.
- There are still a few open comments, from your other mails, I still
have to work upon. So I will work on those in the next version.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachment | Content-Type | Size |
---|---|---|
v27.tar | application/x-tar | 330.5 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Masahiko Sawada | 2020-06-12 06:09:39 | Re: Internal key management system |
Previous Message | Masahiko Sawada | 2020-06-12 05:50:12 | Re: Resetting spilled txn statistics in pg_stat_replication |