Re: [PoC] Reducing planning time when tables have many partitions

From: Thom Brown <thom(at)linux(dot)com>
To: David Rowley <dgrowleyml(at)gmail(dot)com>
Cc: Yuya Watari <watari(dot)yuya(at)gmail(dot)com>, Andrey Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Zhang Mingli <zmlpostgres(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: [PoC] Reducing planning time when tables have many partitions
Date: 2022-12-06 11:16:01
Message-ID: CAA-aLv5pYH92LVmE8r7bD4ABHr=gF_OJC+5NJVo-oH6FGDUMEQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, 5 Dec 2022 at 21:28, David Rowley <dgrowleyml(at)gmail(dot)com> wrote:
>
> On Tue, 6 Dec 2022 at 04:45, Thom Brown <thom(at)linux(dot)com> wrote:
> > Testing your patches with the same 1024 partitions, each with 64
> > sub-partitions, I get a planning time of 205.020 ms, which is now a
> > 1,377x speedup. This has essentially reduced the planning time from a
> > catastrophe to a complete non-issue. Huge win!
>
> Thanks for testing the v10 patches.
>
> I wouldn't have expected such additional gains from v10. I was mostly
> focused on trying to minimise any performance regression for simple
> queries that wouldn't benefit from indexing the EquivalenceMembers.
> Your query sounds like it does not fit into that category. Perhaps it
> is down to the fact that v9-0002 or v9-0003 reverts a couple of the
> optimisations that is causing v9 to be slower than v10 for your query.
> It's hard to tell without more details of what you're running.

I celebrated prematurely as I neglected to wait for the 6th execution
of the prepared statement, which shows the real result. With the v10
patches, it takes 5632.040 ms, a speedup of 50x.

Testing the v9 patches, the same query takes 3388.173 ms, a speedup of
83x. And re-testing v8, I'm getting roughly the same times. These
are all with a cold cache.

So the result isn't as dramatic as I had initially interpreted it to
have unfortunately.

--
Thom

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bharath Rupireddy 2022-12-06 11:16:45 Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures
Previous Message Amit Kapila 2022-12-06 11:10:32 Re: Time delayed LR (WAS Re: logical replication restrictions)