Re: Optimize planner memory consumption for huge arrays

Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> writes:
> On 2/25/24 00:07, Tom Lane wrote:
>> ... I'm not sure if it'd be worth extending
>> the mcxt.c API to provide something like "MemoryContextResetIfBig",
>> with some internal rule that would be cheap to apply like "reset
>> if we have any non-keeper blocks".

> I think MemoryContextResetIfBig is an interesting idea - I think a good
> way to define "big" would be "has multiple blocks", because that's the
> only case where we can actually reclaim some memory.

Yeah. Also: once we had such an idea, it'd be very tempting to apply
it to other frequently-reset contexts like the executor's per-tuple
evaluation contexts. I'm not quite prepared to argue that
MemoryContextReset should just act that way all the time ... but
it's sure interesting to think about.

Another question is whether this wouldn't hurt debugging, in that
dangling-pointer bugs would become harder to catch. We'd certainly
want to turn off the optimization in USE_VALGRIND builds, and maybe
we just shouldn't do it at all if USE_ASSERT_CHECKING.

regards, tom lane