Re: DO with a large amount of statements get stuck with high memory consumption

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Merlin Moncure <mmoncure(at)gmail(dot)com>
Cc: Jan Wieck <jan(at)wi3ck(dot)info>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: DO with a large amount of statements get stuck with high memory consumption
Date: 2016-07-18 14:28:17
Message-ID: 22342.1468852097@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Merlin Moncure <mmoncure(at)gmail(dot)com> writes:
> Hm, maybe, instead of trying to figure out if in a loop, set a
> 'called' flag with each statement and only cache when touched the
> second time. (If that's easier, dunno).

Well, then you just guarantee to lose once. I think Jan's sketch of
marking potentially-cacheable expressions at compile time sounds
promising. I'm envisioning a counter that starts at 1 normally or 0 in a
DO block, increment at beginning of parsing a loop construct, decrement at
end; then mark expressions/statements as cacheable if counter>0.

Of course the next question is what exactly to do differently for a
noncacheable expression. My recollection is that that's all tied
pretty tightly to the plancache these days, so it might take a little
work.

regards, tom lane

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2016-07-18 14:32:02 Re: rethinking dense_alloc (HashJoin) as a memory context
Previous Message Greg Stark 2016-07-18 14:19:18 Re: rethinking dense_alloc (HashJoin) as a memory context