pgsql: Use extended stats for precise estimation of bucket size in hash

From: Alexander Korotkov <akorotkov(at)postgresql(dot)org>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Use extended stats for precise estimation of bucket size in hash
Date: 2025-03-10 11:47:56
Message-ID: E1trbbw-001knB-1S@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Use extended stats for precise estimation of bucket size in hash join

Recognizing the real-life complexity where columns in the table often have
functional dependencies, PostgreSQL's estimation of the number of distinct
values over a set of columns can be underestimated (or much rarely,
overestimated) when dealing with multi-clause JOIN. In the case of hash
join, it can end up with a small number of predicted hash buckets and, as
a result, picking non-optimal merge join.

To improve the situation, we introduce one additional stage of bucket size
estimation - having two or more join clauses estimator lookup for extended
statistics and use it for multicolumn estimation. Clauses are grouped into
lists, each containing expressions referencing the same relation. The result
of the multicolumn estimation made over such a list is combined with others
according to the caller's logic. Clauses that are not estimated are returned
to the caller for further estimation.

Discussion: https://postgr.es/m/52257607-57f6-850d-399a-ec33a654457b%40postgrespro.ru
Author: Andrei Lepikhov <lepihov(at)gmail(dot)com>
Reviewed-by: Andy Fan <zhihui(dot)fan1213(at)gmail(dot)com>
Reviewed-by: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
Reviewed-by: Alena Rybakina <lena(dot)ribackina(at)yandex(dot)ru>
Reviewed-by: Alexander Korotkov <aekorotkov(at)gmail(dot)com>

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/6bb6a62f3cc45624c601d5270673a17447734629

Modified Files
--------------
src/backend/optimizer/path/costsize.c | 12 ++-
src/backend/utils/adt/selfuncs.c | 175 ++++++++++++++++++++++++++++++++
src/include/utils/selfuncs.h | 4 +
src/test/regress/expected/stats_ext.out | 45 ++++++++
src/test/regress/sql/stats_ext.sql | 29 ++++++
5 files changed, 264 insertions(+), 1 deletion(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Tom Lane 2025-03-10 14:23:08 pgsql: Doc: improve description of window function processing.
Previous Message Álvaro Herrera 2025-03-10 11:12:33 Re: pgsql: reindexdb: Add the index-level REINDEX with multiple jobs