Unexpected (wrong?) result querying boolean partitioned table with NULL partition

From: David Kimura <david(dot)g(dot)kimura(at)gmail(dot)com>
To: PostgreSQL Development <pgsql-hackers(at)postgresql(dot)org>
Subject: Unexpected (wrong?) result querying boolean partitioned table with NULL partition
Date: 2023-04-11 21:28:32
Message-ID: CAHnPFjQ5qxs6J_p+g8=ww7GQvfn71_JE+Tygj0S7RdRci1uwPw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Hackers,

Is it fair to assume that, given the same data, a partitioned table should
return the same results as a non-partitioned table? If that's true, then I
think I may have stumbled across a case of wrong results on boolean partitioned
tables.

In following example, I think we incorrectly skip the default partition scan:

CREATE TABLE boolpart (a bool) PARTITION BY LIST (a);
CREATE TABLE boolpart_default PARTITION OF boolpart default;
CREATE TABLE boolpart_t PARTITION OF boolpart FOR VALUES IN ('true');
CREATE TABLE boolpart_f PARTITION OF boolpart FOR VALUES IN ('false');
INSERT INTO boolpart VALUES (true), (false), (null);

EXPLAIN SELECT * FROM boolpart WHERE a IS NOT true;
QUERY PLAN
-----------------------------------------------------------------------
Seq Scan on boolpart_f boolpart (cost=0.00..38.10 rows=1405 width=1)
Filter: (a IS NOT TRUE)
(2 rows)

SELECT * FROM boolpart WHERE a IS NOT true;
a
---
f
(1 row)

Compare that to the result of a non-partitioned table:

CREATE TABLE booltab (a bool);
INSERT INTO booltab VALUES (true), (false), (null);

EXPLAIN SELECT * FROM booltab WHERE a IS NOT true;
QUERY PLAN
-----------------------------------------------------------
Seq Scan on booltab (cost=0.00..38.10 rows=1405 width=1)
Filter: (a IS NOT TRUE)
(2 rows)

SELECT * FROM booltab WHERE a IS NOT true;
a
---
f

(2 rows)

I think the issue has to do with assumptions made about boolean test IS NOT
inequality logic which is different from inequality of other operators.
Specifically, "true IS NOT NULL" is not the same as "true<>NULL".

In partition pruning, match_boolean_partition_clause() tries to match partkey
with clause and outputs PARTCLAUSE_MATCH_CLAUSE and an outconst TRUE for
(IS_TRUE or IS_NOT_FALSE) and inversely FALSE for (IS_FALSE or IS_NOT_TRUE).
However, I don't think this gradularity is sufficient for "IS NOT" logic when a
NULL value partition is present.

One idea is to use the negation operator for IS_NOT_(true|false) (i.e.
BooleanNotEqualOperator instead of BooleanEqualOperator). But besides
presumably being a more expensive operation, not equal is not part of the btree
opfamily for bool_ops. So, seems like that won't really fit into the current
partition pruning framework.

Then I realized that the issue is just about adding the default or null
partition in these very particular scenarios. And struct PartitionBoundInfoData
already holds that information. So if we can identify these scenarios and pass
that information into get_matching_partitions() then we can add the necessary
partitions. Attached is a very rough sketch of that idea.

Thoughts? Does this seem like a legit issue? And if so, do either of the
proposed solutions seem reasonable?

Thanks,
David

Attachment Content-Type Size
boolpart_scan_nullpart.patch application/octet-stream 5.7 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Melanie Plageman 2023-04-11 21:29:40 Re: Show various offset arrays for heap WAL records
Previous Message Sandro Santilli 2023-04-11 21:27:37 Re: [PATCH] Support % wildcard in extension upgrade filenames