Re: Excessive memory usage in multi-statement queries w/ partitioning

From: Julien Rouhaud <rjuju123(at)gmail(dot)com>
To: Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Joe Conway <mail(at)joeconway(dot)com>, David Rowley <david(dot)rowley(at)2ndquadrant(dot)com>, Andreas Seltenreich <andreas(dot)seltenreich(at)credativ(dot)de>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Excessive memory usage in multi-statement queries w/ partitioning
Date: 2019-07-04 09:52:22
Message-ID: CAOBaU_ZAQiJpZ0YRoe+zUnJ0d1+Yd656oRtcSRQvx+WmNiWE7Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, May 28, 2019 at 6:57 AM Amit Langote
<Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp> wrote:
>
> On 2019/05/27 21:56, Tom Lane wrote:
> > Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp> writes:
> >> On 2019/05/24 23:28, Tom Lane wrote:
> >>> So my thought, if we want to do something about this, is not "find
> >>> some things we can pfree at the end of planning" but "find a way
> >>> to use a separate context for each statement in the query string".
> >
> >> Maybe like the attached? I'm not sure if we need to likewise be concerned
> >> about exec_sql_string() being handed multi-query strings.

the whole extension sql script is passed to execute_sql_string(), so I
think that it's a good thing to have similar workaround there.

About the patch:

- * Switch to appropriate context for constructing querytrees (again,
- * these must outlive the execution context).
+ * Switch to appropriate context for constructing querytrees.
+ * Memory allocated during this construction is released before
+ * the generated plan is executed.

The comment should mention query and plan trees, everything else seems ok to me.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message tushar 2019-07-04 10:22:21 Re: Minimal logical decoding on standbys
Previous Message Andrey Borodin 2019-07-04 09:10:24 Re: Optimize partial TOAST decompression