pgsql: Compress GIN posting lists, for smaller index size.

From: Heikki Linnakangas <heikki(dot)linnakangas(at)iki(dot)fi>
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: Compress GIN posting lists, for smaller index size.
Date: 2014-01-22 17:28:48
Message-ID: E1W61bc-0008Tu-4w@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers pgsql-hackers

Compress GIN posting lists, for smaller index size.

GIN posting lists are now encoded using varbyte-encoding, which allows them
to fit in much smaller space than the straight ItemPointer array format used
before. The new encoding is used for both the lists stored in-line in entry
tree items, and in posting tree leaf pages.

To maintain backwards-compatibility and keep pg_upgrade working, the code
can still read old-style pages and tuples. Posting tree leaf pages in the
new format are flagged with GIN_COMPRESSED flag, to distinguish old and new
format pages. Likewise, entry tree tuples in the new format have a
GIN_ITUP_COMPRESSED flag set in a bit that was previously unused.

This patch bumps GIN_CURRENT_VERSION from 1 to 2. New indexes created with
version 9.4 will therefore have version number 2 in the metapage, while old
pg_upgraded indexes will have version 1. The code treats them the same, but
it might be come handy in the future, if we want to drop support for the
uncompressed format.

Alexander Korotkov and me. Reviewed by Tomas Vondra and Amit Langote.

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/36a35c550ac114caa423bcbe339d3515db0cd957

Modified Files
--------------
contrib/pgstattuple/expected/pgstattuple.out | 2 +-
src/backend/access/gin/README | 123 ++-
src/backend/access/gin/ginbtree.c | 73 +-
src/backend/access/gin/gindatapage.c | 1450 ++++++++++++++++++++------
src/backend/access/gin/ginentrypage.c | 134 ++-
src/backend/access/gin/ginfast.c | 2 +-
src/backend/access/gin/ginget.c | 117 ++-
src/backend/access/gin/gininsert.c | 67 +-
src/backend/access/gin/ginpostinglist.c | 386 ++++++-
src/backend/access/gin/ginvacuum.c | 232 +++--
src/backend/access/gin/ginxlog.c | 184 ++--
src/backend/access/rmgrdesc/gindesc.c | 45 +-
src/include/access/gin_private.h | 212 +++-
13 files changed, 2309 insertions(+), 718 deletions(-)

Responses

Browse pgsql-committers by date

  From Date Subject
Next Message Fujii Masao 2014-01-22 17:44:35 Re: pgsql: Compress GIN posting lists, for smaller index size.
Previous Message Andrew Dunstan 2014-01-22 13:48:04 pgsql: Reindent json.c and jsonfuncs.c.

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2014-01-22 17:40:34 Re: dynamic shared memory and locks
Previous Message Andres Freund 2014-01-22 17:23:02 Re: Incorrectly reporting config errors