Re: [PoC] Reducing planning time when tables have many partitions

From: David Rowley <dgrowleyml(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Amit Langote <amitlangote09(at)gmail(dot)com>, Yuya Watari <watari(dot)yuya(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, jian he <jian(dot)universality(at)gmail(dot)com>, Alena Rybakina <lena(dot)ribackina(at)yandex(dot)ru>, Andrei Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru>, Thom Brown <thom(at)linux(dot)com>, Zhang Mingli <zmlpostgres(at)gmail(dot)com>
Subject: Re: [PoC] Reducing planning time when tables have many partitions
Date: 2025-04-06 23:04:29
Message-ID: CAApHDvrWWe2_AWhhJ3LK9-3GwLG=rx+7VpDw=gnLGB7W7K-hUg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sat, 5 Apr 2025 at 16:55, David Rowley <dgrowleyml(at)gmail(dot)com> wrote:
> I am still thinking about the duplicate members being returned from
> the iterator for child join rels due to them being duplicated into
> each component relid element in ec_childmembers. I did consider if
> these could just not be duplicated and instead just put into the
> ec_childmember element according to their lowest component relid. For
> that to work, all callers that need these would need to ensure they
> never pass some subset of child_relids when setting up the
> EquivalenceMemberIterator. I need to study a bit more to understand if
> that's doable.

It looks like the child members added by
add_child_join_rel_equivalences() are only required for pathkey
requirements. The EquivalenceMember mentions:

* for an appendrel child. These members are used for determining the
* pathkeys of scans on the child relation and for explicitly sorting the
* child when necessary to build a MergeAppend path for the whole appendrel
* tree. An em_is_child member has no impact on the properties of the EC as a

I used the attached .txt file to highlight the places where the
iterator returned the same member twice and saw only that
find_ec_member_matching_expr() does this.

Because during createplan, we'll have the specific RelOptInfo that we
need the EquivalenceMember for, we'll be passing the "relids" of that
RelOptInfo to setup_eclass_member_iterator(), in which case, I think
it's fine to store the member in just one of the ec_childmembers[]
array slots for just one of the relids making up the
RELOPT_OTHER_JOINREL's component relids as that means it'll be found
once only due to how eclass_member_iterator_next() looks at all of the
ec_childmembers[] elements for the given relids.

Doing this also allows the code in add_child_eq_member() to be simplified.

I made this happen in the attached v41 patch, and that's the last
outstanding issue that I had for this.

I think this is worthy of getting into v18. Does anyone else think
differently? It'd be good to know that soon.

David

Attachment Content-Type Size
eq_member_find_duplicates.patch.txt text/plain 2.5 KB
v41-0001-Speedup-child-EquivalenceMember-lookup-in-planne.patch application/octet-stream 44.0 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Chapman Flack 2025-04-06 23:27:22 Re: FmgrInfo allocation patterns (and PL handling as staged programming)
Previous Message Tom Lane 2025-04-06 22:53:14 Re: Logging which local address was connected to in log_line_prefix