Re: Memory consumed by paths during partitionwise join planning

On 6/2/2024 19:51, Ashutosh Bapat wrote:
> On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat
> The patches are raw. make check has some crashes that I need to fix. I
> am waiting to hear whether this is useful and whether the design is on
> the right track.
Let me write words of opinion on that feature.
I generally like the idea of a refcount field. We had problems
generating alternative paths in extensions and designing the Asymmetric
Join feature [1] when we proposed an alternative path one level below
and called the add_path() routine. We had lost the path in the path list
referenced from the upper RelOptInfo. This approach allows us to make
more handy solutions instead of hacking with a copy/restore pathlist.
But I'm not sure about freeing unreferenced paths. I would have to see
alternatives in the pathlist.
About partitioning. As I discovered planning issues connected to
partitions, the painful problem is a rule, according to which we are
trying to use all nomenclature of possible paths for each partition.
With indexes, it quickly increases optimization work. IMO, this can help
a 'symmetrical' approach, which could restrict the scope of possible
pathways for upcoming partitions if we filter some paths in a set of
previously planned partitions.
Also, I am glad to see a positive opinion about the path_walker()
routine. Somewhere else, for example, in [2], it seems we need it too.

[1]
https://www.postgresql.org/message-id/flat/CAOP8fzaVL_2SCJayLL9kj5pCA46PJOXXjuei6-3aFUV45j4LJQ%40mail.gmail.com
[2]
https://www.postgresql.org/message-id/flat/CAMbWs496%2BN%3DUAjOc%3DrcD3P7B6oJe4rZw08e_TZRUsWbPxZW3Tw%40mail.gmail.com
--
regards,
Andrei Lepikhov
Postgres Professional