From: | Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> |
---|---|
To: | Amit Langote <amitlangote09(at)gmail(dot)com> |
Cc: | Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, tomas(at)vondra(dot)me, vignesh C <vignesh21(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Richard Guo <guofenglinux(at)gmail(dot)com>, David Rowley <dgrowleyml(at)gmail(dot)com> |
Subject: | Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning |
Date: | 2025-03-27 10:27:35 |
Message-ID: | CAExHW5vjRj_qRSn0wGbjs3S8P7cETBD9raoZkhPoV=q7n8u_Mw@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi,
On Wed, Mar 26, 2025 at 4:59 PM Ashutosh Bapat
<ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
>
>
> The averages are now more stable than the previous exercise. Some regressions seen with the first exercise are not seen with the other and some improvements seens with the first exercise are not seen with the second and vice versa. The regressions present in both the exercises will not be seen, if I repeat the exercise a few more times. So I think those regressions or improvements seen with lower number of joins and lower number of partitions aren't real and they are mostly within noise level. However, the improvements seen with higher numbers of joins and partitions are always there irrespective of the number of times I repeat the exercise. Please note that we have only tried partitions upto 256. Previous measurements have seen stable improvements in case of higher number of partitions.
>
Further, I experimented with hash table size. Attached images have
four graphs for planning time and planner's memory consumption
measured for a 3-way join for initial has table sizes of 64, 128 and
256 respectively.
1. Planning time vs number of partitions with PWJ enabled
2. planning time vs number of partitions with PWJ disabled
3. Memory consumed vs number of partitions with PWJ enabled
4. Memory consumed vs number of partitions with PWJ disabled
Also find attached the spreadsheet containing the measurements and
also the charts.
In the graphs, one can observe that the lines corresponding to all
three hash table sizes are inseparable. That leads to a conclusion
that the planning time or memory consumption doesn't change with hash
table size.
As a result I am keeping the initial hash table size = 256L, same as
the previously attached patches.
--
Best Wishes,
Ashutosh Bapat
Attachment | Content-Type | Size |
---|---|---|
![]() |
image/png | 29.9 KB |
![]() |
image/png | 25.6 KB |
![]() |
image/png | 28.3 KB |
EffectOfHashTableSize27Mar2025.ods | application/vnd.oasis.opendocument.spreadsheet | 51.3 KB |
![]() |
image/png | 27.4 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Andrew Dunstan | 2025-03-27 10:46:50 | Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote |
Previous Message | Andrea Gelmini | 2025-03-27 10:25:25 | Re: FileFallocate misbehaving on XFS |