From: | "Lepikhov Andrei" <a(dot)lepikhov(at)postgrespro(dot)ru> |
---|---|
To: | pgsql-hackers(at)lists(dot)postgresql(dot)org |
Cc: | Евгений Бредня <e(dot)brednya(at)postgrespro(dot)ru> |
Subject: | Optimize planner memory consumption for huge arrays |
Date: | 2023-09-04 05:25:44 |
Message-ID: | em9939439a-441a-4b27-a977-ebdf5987dc49@7d14f008.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi, hackers,
Looking at the planner behaviour with the memory consumption patch [1], I figured out that arrays increase memory consumption by the optimizer significantly. See init.sql in attachment.
The point here is that the planner does small memory allocations for each element during estimation. As a result, it looks like the planner consumes about 250 bytes for each integer element.
It is maybe not a problem most of the time. However, in the case of partitions, memory consumption multiplies by each partition. Such a corner case looks weird, but the fix is simple. So, why not?
The diff in the attachment is proof of concept showing how to reduce wasting of memory. Having benchmarked a bit, I didn't find any overhead.
[1] Report planning memory in EXPLAIN ANALYZE
https://www.postgresql.org/message-id/flat/CAExHW5sZA=5LJ_ZPpRO-w09ck8z9p7eaYAqq3Ks9GDfhrxeWBw(at)mail(dot)gmail(dot)com
--
Regards,
Andrey Lepikhov
Attachment | Content-Type | Size |
---|---|---|
init.sql | application/sql | 344 bytes |
array_compact_memusage.diff | application/octet-stream | 1.5 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Amit Kapila | 2023-09-04 05:48:24 | Re: Impact of checkpointer during pg_upgrade |
Previous Message | Dilip Kumar | 2023-09-04 05:03:20 | Re: Impact of checkpointer during pg_upgrade |