From: | Robert Haas <rhaas(at)postgresql(dot)org> |
---|---|
To: | pgsql-committers(at)postgresql(dot)org |
Subject: | pgsql: Support parallel btree index builds. |
Date: | 2018-02-02 18:37:11 |
Message-ID: | E1ehgD1-0008TG-4S@gemulon.postgresql.org |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-committers pgsql-hackers |
Support parallel btree index builds.
To make this work, tuplesort.c and logtape.c must also support
parallelism, so this patch adds that infrastructure and then applies
it to the particular case of parallel btree index builds. Testing
to date shows that this can often be 2-3x faster than a serial
index build.
The model for deciding how many workers to use is fairly primitive
at present, but it's better than not having the feature. We can
refine it as we get more experience.
Peter Geoghegan with some help from Rushabh Lathia. While Heikki
Linnakangas is not an author of this patch, he wrote other patches
without which this feature would not have been possible, and
therefore the release notes should possibly credit him as an author
of this feature. Reviewed by Claudio Freire, Heikki Linnakangas,
Thomas Munro, Tels, Amit Kapila, me.
Discussion: http://postgr.es/m/CAM3SWZQKM=Pzc=CAHzRixKjp2eO5Q0Jg1SoFQqeXFQ647JiwqQ@mail.gmail.com
Discussion: http://postgr.es/m/CAH2-Wz=AxWqDoVvGU7dq856S4r6sJAj6DBn7VMtigkB33N5eyg@mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/9da0cc35284bdbe8d442d732963303ff0e0a40bc
Modified Files
--------------
contrib/bloom/blinsert.c | 3 +-
doc/src/sgml/config.sgml | 44 +-
doc/src/sgml/monitoring.sgml | 12 +-
doc/src/sgml/ref/create_index.sgml | 58 ++
doc/src/sgml/ref/create_table.sgml | 4 +-
src/backend/access/brin/brin.c | 4 +-
src/backend/access/gin/gininsert.c | 2 +-
src/backend/access/gist/gistbuild.c | 2 +-
src/backend/access/hash/hash.c | 2 +-
src/backend/access/hash/hashsort.c | 1 +
src/backend/access/heap/heapam.c | 28 +-
src/backend/access/nbtree/nbtree.c | 134 +---
src/backend/access/nbtree/nbtsort.c | 878 +++++++++++++++++++++++++-
src/backend/access/spgist/spginsert.c | 3 +-
src/backend/access/transam/parallel.c | 12 +-
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 2 +-
src/backend/catalog/index.c | 123 +++-
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 3 +-
src/backend/commands/indexcmds.c | 7 +-
src/backend/executor/execParallel.c | 2 +-
src/backend/executor/nodeAgg.c | 6 +-
src/backend/executor/nodeSort.c | 2 +-
src/backend/optimizer/path/allpaths.c | 18 +-
src/backend/optimizer/path/costsize.c | 4 +-
src/backend/optimizer/plan/planner.c | 136 ++++
src/backend/postmaster/pgstat.c | 3 +
src/backend/storage/file/buffile.c | 61 +-
src/backend/storage/file/fd.c | 10 +
src/backend/utils/adt/orderedsetaggs.c | 2 +
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc.c | 10 +
src/backend/utils/misc/postgresql.conf.sample | 3 +-
src/backend/utils/probes.d | 2 +-
src/backend/utils/sort/logtape.c | 199 +++++-
src/backend/utils/sort/tuplesort.c | 595 ++++++++++++++---
src/include/access/nbtree.h | 14 +-
src/include/access/parallel.h | 4 +-
src/include/access/relscan.h | 1 +
src/include/catalog/index.h | 9 +-
src/include/miscadmin.h | 1 +
src/include/nodes/execnodes.h | 6 +-
src/include/optimizer/paths.h | 2 +-
src/include/optimizer/planner.h | 1 +
src/include/pgstat.h | 1 +
src/include/storage/buffile.h | 2 +
src/include/storage/fd.h | 1 +
src/include/utils/logtape.h | 39 +-
src/include/utils/tuplesort.h | 132 +++-
src/tools/pgindent/typedefs.list | 6 +
51 files changed, 2237 insertions(+), 361 deletions(-)
From | Date | Subject | |
---|---|---|---|
Next Message | Peter Eisentraut | 2018-02-02 20:07:10 | pgsql: Fix application of identity values in some cases |
Previous Message | Tom Lane | 2018-02-02 14:52:32 | Re: pgsql: Remove byte-masking macros for Datum conversion macros |
From | Date | Subject | |
---|---|---|---|
Next Message | Robert Haas | 2018-02-02 18:37:13 | Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation) |
Previous Message | Oliver Ford | 2018-02-02 17:31:32 | Re: Add RANGE with values and exclusions clauses to the Window Functions |