From: | Thomas Munro <tmunro(at)postgresql(dot)org> |
---|---|
To: | pgsql-committers(at)lists(dot)postgresql(dot)org |
Subject: | pgsql: Rotate instead of shifting hash join batch number. |
Date: | 2019-12-24 00:23:21 |
Message-ID: | E1ijXyr-0006ZP-Q7@gemulon.postgresql.org |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-committers |
Rotate instead of shifting hash join batch number.
Our algorithm for choosing batch numbers turned out not to work
effectively for multi-billion key inner relations. We would use
more hash bits than we have, and effectively concentrate all tuples
into a smaller number of batches than we intended. While ideally
we should switch to wider hashes, for now, change the algorithm to
one that effectively gives up bits from the bucket number when we
don't have enough bits. That means we'll finish up with longer
bucket chains than would be ideal, but that's better than having
batches that don't fit in work_mem and can't be divided.
Batch-patch to all supported releases.
Author: Thomas Munro
Reviewed-by: Tom Lane, thanks also to Tomas Vondra, Alvaro Herrera, Andres Freund for testing and discussion
Reported-by: James Coleman
Discussion: https://postgr.es/m/16104-dc11ed911f1ab9df%40postgresql.org
Branch
------
REL9_4_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/5c0a132cf1410fb9dea577e6b84b8560bb0f7e03
Modified Files
--------------
src/backend/executor/nodeHash.c | 21 +++++++++++++++++----
1 file changed, 17 insertions(+), 4 deletions(-)
From | Date | Subject | |
---|---|---|---|
Next Message | Michael Paquier | 2019-12-24 03:22:04 | pgsql: Replace use of strerror() with %s by %m in pg_waldump |
Previous Message | Thomas Munro | 2019-12-24 00:23:07 | pgsql: Rotate instead of shifting hash join batch number. |