From: | Amit Langote <amitlangote09(at)gmail(dot)com> |
---|---|
To: | Julien Rouhaud <rjuju123(at)gmail(dot)com> |
Cc: | Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Joe Conway <mail(at)joeconway(dot)com>, David Rowley <david(dot)rowley(at)2ndquadrant(dot)com>, Andreas Seltenreich <andreas(dot)seltenreich(at)credativ(dot)de>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: Excessive memory usage in multi-statement queries w/ partitioning |
Date: | 2019-07-08 08:52:20 |
Message-ID: | CA+HiwqEwK-rZMt3TY_FL4jPKOSFJhH_FzAdW8XXNZx4Er753sQ@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi Julien,
Thanks for taking a look at this.
On Thu, Jul 4, 2019 at 6:52 PM Julien Rouhaud <rjuju123(at)gmail(dot)com> wrote:
> On Tue, May 28, 2019 at 6:57 AM Amit Langote wrote:
> > >> Maybe like the attached? I'm not sure if we need to likewise be concerned
> > >> about exec_sql_string() being handed multi-query strings.
>
> the whole extension sql script is passed to execute_sql_string(), so I
> think that it's a good thing to have similar workaround there.
That makes sense, although it is perhaps much less likely for memory
usage explosion to occur in execute_sql_strings(), because the scripts
passed to execute_sql_strings() mostly contain utility statements and
rarely anything whose planning will explode in memory usage.
Anyway, I've added similar handling in execute_sql_strings() for consistency.
Now I wonder if we'll need to consider another path which calls
pg_plan_queries() on a possibly multi-statement query --
BuildCachedPlan()...
> About the patch:
>
> - * Switch to appropriate context for constructing querytrees (again,
> - * these must outlive the execution context).
> + * Switch to appropriate context for constructing querytrees.
> + * Memory allocated during this construction is released before
> + * the generated plan is executed.
>
> The comment should mention query and plan trees, everything else seems ok to me.
Okay, fixed.
Attached updated patch. Thanks again.
Regards,
Amit
Attachment | Content-Type | Size |
---|---|---|
parse-plan-memcxt_v2.patch | application/octet-stream | 3.3 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Masahiko Sawada | 2019-07-08 09:04:28 | Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS) |
Previous Message | kuroda.hayato@fujitsu.com | 2019-07-08 08:27:52 | RE: [PATCH] memory leak in ecpglib |