Re: [PoC] Reducing planning time when tables have many partitions

From: Yuya Watari <watari(dot)yuya(at)gmail(dot)com>
To: David Rowley <dgrowleyml(at)gmail(dot)com>
Cc: Thom Brown <thom(at)linux(dot)com>, Andrey Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Zhang Mingli <zmlpostgres(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: [PoC] Reducing planning time when tables have many partitions
Date: 2023-01-30 10:02:37
Message-ID: CAJ2pMkb2LhBR0N9nPdte62EFp6R5PV8PFkDb0nyoPCNqVAbZqQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello,

On Fri, Jan 27, 2023 at 12:48 PM Yuya Watari <watari(dot)yuya(at)gmail(dot)com> wrote:
> However, there is more bad news. Unfortunately, some regression tests
> are failing in my environment. I'm not sure why, but it could be that
> a) my sanity checking code (v12-0004) is wrong, or b) our patches have
> some bugs.
>
> I will investigate this issue further, and share the results when found.

I have investigated this issue and concluded that b) our patches have
some bugs. I have attached the modified patches to this email. This
version passed regression tests in my environment.

1. v13-0005

The first bug is in eclass_member_iterator_strict_next(). As I
mentioned in the commit message, the original code incorrectly missed
EquivalenceMembers with empty em_relids when 'with_norel_members' is
true.

I show my changes as follows:

===
- if (!iter->with_children && em->em_is_child)
- continue;

- if (!iter->with_norel_members && bms_is_empty(em->em_relids))
- continue;

- if (!bms_is_subset(iter->with_relids, em->em_relids))
- continue;

- iter->current_index = foreach_current_index(lc);
+ if ((iter->with_norel_members && bms_is_empty(em->em_relids))
+ || (bms_is_subset(iter->with_relids, em->em_relids)
+ && (iter->with_children || !em->em_is_child)))
+ {
+ iter->current_index = foreach_current_index(lc);
===

EquivalenceMembers with empty em_relids will pass the second 'if'
condition when 'with_norel_members' is true. These members should be
returned. However, since the empty em_relids can never be superset of
any non-empty relids, the EMs may fail the last condition. Therefore,
the original code missed some members.

2. v13-0006

The second bug exists in get_ecmember_indexes_strict(). As I described
in the comment, if the empty relids is given, this function must
return all members because their em_relids are always superset. I am
concerned that this change may adversely affect performance.
Currently, I have not seen any degradation.

3. v13-0007

The last one is in add_eq_member(). I am not sure why this change is
working, but it is probably related to the concerns David mentioned in
the previous mail. The v13-0007 may be wrong, so it should be
reconsidered.

--
Best regards,
Yuya Watari

Attachment Content-Type Size
v13-0001-Add-Bitmapset-indexes-for-faster-lookup-of-Equiv.patch application/octet-stream 81.6 KB
v13-0002-Adjust-bms_int_members-so-that-it-shortens-the-l.patch application/octet-stream 2.5 KB
v13-0003-Add-iterators-for-looping-over-EquivalenceMember.patch application/octet-stream 22.2 KB
v13-0004-Add-a-validation-to-check-that-two-iteration-res.patch application/octet-stream 4.2 KB
v13-0005-Fix-incorrect-filtering-condition-in-iteration.patch application/octet-stream 3.6 KB
v13-0006-Fix-a-bug-with-get_ecmember_indexes_strict-incor.patch application/octet-stream 1.1 KB
v13-0007-Fix-a-bug.patch application/octet-stream 904 bytes

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Takamichi Osumi (Fujitsu) 2023-01-30 10:04:59 RE: Time delayed LR (WAS Re: logical replication restrictions)
Previous Message Richard Guo 2023-01-30 09:56:38 Re: Making Vars outer-join aware