pgsql: Allow index AMs to cache data across aminsert calls within a SQL

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: Allow index AMs to cache data across aminsert calls within a SQL
Date: 2017-02-09 16:52:24
Message-ID: E1cbrxI-0000Gf-Fr@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Allow index AMs to cache data across aminsert calls within a SQL command.

It's always been possible for index AMs to cache data across successive
amgettuple calls within a single SQL command: the IndexScanDesc.opaque
field is meant for precisely that. However, no comparable facility
exists for amortizing setup work across successive aminsert calls.
This patch adds such a feature and teaches GIN, GIST, and BRIN to use it
to amortize catalog lookups they'd previously been doing on every call.
(The other standard index AMs keep everything they need in the relcache,
so there's little to improve there.)

For GIN, the overall improvement in a statement that inserts many rows
can be as much as 10%, though it seems a bit less for the other two.
In addition, this makes a really significant difference in runtime
for CLOBBER_CACHE_ALWAYS tests, since in those builds the repeated
catalog lookups are vastly more expensive.

The reason this has been hard up to now is that the aminsert function is
not passed any useful place to cache per-statement data. What I chose to
do is to add suitable fields to struct IndexInfo and pass that to aminsert.
That's not widening the index AM API very much because IndexInfo is already
within the ken of ambuild; in fact, by passing the same info to aminsert
as to ambuild, this is really removing an inconsistency in the AM API.

Discussion: https://postgr.es/m/27568.1486508680@sss.pgh.pa.us

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/86d911ec0f9d4643e9a47db42510959dec0ed76b

Modified Files
--------------
contrib/bloom/blinsert.c | 4 +++-
contrib/bloom/bloom.h | 3 ++-
doc/src/sgml/indexam.sgml | 11 ++++++++++-
src/backend/access/brin/brin.c | 25 +++++++++++++++----------
src/backend/access/gin/gininsert.c | 27 ++++++++++++++++++---------
src/backend/access/gist/gist.c | 25 ++++++++++++++-----------
src/backend/access/hash/hash.c | 3 ++-
src/backend/access/heap/tuptoaster.c | 7 +++++--
src/backend/access/index/indexam.c | 5 +++--
src/backend/access/nbtree/nbtree.c | 3 ++-
src/backend/access/spgist/spginsert.c | 3 ++-
src/backend/catalog/index.c | 7 ++++++-
src/backend/catalog/indexing.c | 3 ++-
src/backend/catalog/toasting.c | 2 ++
src/backend/commands/constraint.c | 3 ++-
src/backend/commands/indexcmds.c | 4 ++++
src/backend/executor/execIndexing.c | 3 ++-
src/include/access/amapi.h | 3 ++-
src/include/access/brin_internal.h | 3 ++-
src/include/access/genam.h | 6 +++++-
src/include/access/gin_private.h | 3 ++-
src/include/access/gist_private.h | 3 ++-
src/include/access/hash.h | 3 ++-
src/include/access/nbtree.h | 3 ++-
src/include/access/spgist.h | 3 ++-
src/include/nodes/execnodes.h | 4 ++++
26 files changed, 117 insertions(+), 52 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Robert Haas 2017-02-09 19:36:34 pgsql: pageinspect: Fix hash_bitmap_info not to read the underlying pag
Previous Message Andres Freund 2017-02-09 01:06:26 pgsql: Add explicit ORDER BY to a few tests that exercise hash-join cod