Optimization issue of branching UNION ALL

From: Andrey Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Optimization issue of branching UNION ALL
Date: 2022-12-21 04:14:02
Message-ID: 703c09a2-08f3-d2ec-b33d-dbecd62428b8@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Client report on a corner case have shown up possible minor
non-optimality in procedure of transformation of simple UNION ALL
statement tree.
Complaint is about auto-generated query with 1E4 simple union all's (see
t.sh to generate a demo script). The reason: in REL_11_STABLE it is
planned and executed in a second, but REL_12_STABLE and beyond makes
matters worse: planning of such a query needs tons of gigabytes of RAM.

Superficial study revealed possibly unnecessary operations that could be
avoided:
1. Walking across a query by calling substitute_phv_relids() even if
lastPHId shows that no one phv is presented.
2. Iterative passes along the append_rel_list for replacing vars in the
translated_vars field. I can't grasp real necessity of passing all the
append_rel_list during flattening of an union all leaf subquery. No one
can reference this leaf, isn't it?

In attachment you can see some sketch that reduces a number of planner
cycles/copyings.

--
Regards
Andrey Lepikhov
Postgres Professional

Attachment Content-Type Size
t.sh application/x-shellscript 187 bytes
union_all_optimization.patch text/x-patch 3.7 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2022-12-21 04:52:21 Re: Simplifications for error messages related to compression
Previous Message Takamichi Osumi (Fujitsu) 2022-12-21 03:28:01 assertion failures on BuildFarm that happened in slab.c