Re: [PoC] Reducing planning time when tables have many partitions

From: Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>
To: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>
Cc: Yuya Watari <watari(dot)yuya(at)gmail(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, jian he <jian(dot)universality(at)gmail(dot)com>, Alena Rybakina <lena(dot)ribackina(at)yandex(dot)ru>, Andrei Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru>, David Rowley <dgrowleyml(at)gmail(dot)com>, Thom Brown <thom(at)linux(dot)com>, Zhang Mingli <zmlpostgres(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Subject: Re: [PoC] Reducing planning time when tables have many partitions
Date: 2024-12-03 12:10:31
Message-ID: CAExHW5tNMNQ5ennUM3QK+MQGw703M9T6z-LvBKKAcgpJ3KW14Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Dec 3, 2024 at 4:08 PM Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> wrote:
>
> Hello,
>
> On 2024-Dec-03, Ashutosh Bapat wrote:
>
> > For one of the earlier versions, I had reported a large memory
> > consumption in all cases and increase in planning time for Assert
> > enabled builds. How does the latest version perform in those aspects?
>
> I don't think planning time in assert-enabled builds is something we
> should worry about, at all. Planning time in production builds is the
> important one.
>

This was discussed earlier. See a few emails from [1] going backwards.
The degradation was Nx, if I am reading those emails right. That means
somebody who is working with a large number of partitions has to spend
Nx time in running their tests. Given that the planning time with
thousands of partitions is already in seconds, slowing that further
down, even in an assert build is slowing development down further. My
suggestion of using OPTIMIZER_DEBUG will help us keep the sanity
checks and also not slow down development.

[1] https://www.postgresql.org/message-id/CAJ2pMkZrFS8EfvZpkw9CP0iqWk=EaAxzaKWS7dW+FTtqkUOWxA@mail.gmail.com

--
Best Wishes,
Ashutosh Bapat

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2024-12-03 12:14:10 Re: Memory leak in WAL sender with pgoutput (v10~)
Previous Message Amul Sul 2024-12-03 12:00:20 Re: NOT ENFORCED constraint feature