pgsql: Reduce memory used by partitionwise joins

From: Richard Guo <rguo(at)postgresql(dot)org>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Reduce memory used by partitionwise joins
Date: 2024-07-29 02:37:19
Message-ID: E1sYGGF-001gjW-5U@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Reduce memory used by partitionwise joins

In try_partitionwise_join, we aim to break down the join between two
partitioned relations into joins between matching partitions. To
achieve this, we iterate through each pair of partitions from the two
joining relations and create child-join relations for them. With
potentially thousands of partitions, the local objects allocated in
each iteration can accumulate significant memory usage. Therefore, we
opt to eagerly free these local objects at the end of each iteration.

In line with this approach, this patch frees the bitmap set that
represents the relids of child-join relations at the end of each
iteration. Additionally, it modifies build_child_join_rel() to reuse
the AppendRelInfo structures generated within each iteration.

Author: Ashutosh Bapat
Reviewed-by: David Christensen, Richard Guo
Discussion: https://postgr.es/m/CAExHW5s4EqY43oB=ne6B2=-xLgrs9ZGeTr1NXwkGFt2j-OmaQQ@mail.gmail.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/513f4472a4a0d294ca64123627ce7b48ce0ee7c1

Modified Files
--------------
src/backend/optimizer/path/joinrels.c | 15 +++++++++++----
src/backend/optimizer/util/relnode.c | 18 +++---------------
src/include/optimizer/pathnode.h | 3 ++-
3 files changed, 16 insertions(+), 20 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Heikki Linnakangas 2024-07-29 11:27:11 pgsql: Fix outdated comment in smgrtruncate()
Previous Message Jeff Davis 2024-07-29 00:10:01 pgsql: Refactor pg_set_regex_collation() for clarity.