Re: Optimize planner memory consumption for huge arrays

From: Andrei Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru>
To: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>
Cc: pgsql-hackers(at)lists(dot)postgresql(dot)org, Евгений Бредня <e(dot)brednya(at)postgrespro(dot)ru>
Subject: Re: Optimize planner memory consumption for huge arrays
Date: 2024-02-20 04:17:31
Message-ID: c4d5abd2-9831-44b3-b8eb-8ab322bef559@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 19/2/2024 20:47, Tomas Vondra wrote:
> On 9/8/23 07:11, Lepikhov Andrei wrote:
>> Just for comparison, without partitioning:
>> elems 1 1E1 1E2 1E3 1E4
>> master: 12kB 14kB 37kB 266kB 2.5MB
>> patched: 12kB 11.5kB 13kB 24kB 141kB
>>
>
> These improvements look pretty nice, considering how simple the patch
> seems to be. I can't even imagine how much memory we'd need with even
> more partitions (say, 1000) if 100 partitions means 274MB.
>
> BTW when releasing memory in scalararraysel, wouldn't it be good to also
> free the elem_values/elem_nulls? I haven't tried and maybe it's not that
> significant amount.
Agree. Added into the next version of the patch.
Moreover, I see a slight planning speedup. Looking into the reason for
that, I discovered that it is because sometimes the planner utilizes the
same memory piece for the next array element. It finds this piece more
quickly than before that optimization.

--
regards,
Andrei Lepikhov
Postgres Professional

Attachment Content-Type Size
0001-Utilize-memory-in-scalararraysel-in-more-optimal-man.patch text/plain 2.4 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Shlok Kyal 2024-02-20 04:19:18 Re: speed up a logical replica setup
Previous Message jian he 2024-02-20 04:03:31 Re: POC, WIP: OR-clause support for indexes