pgsql: Rewrite the way GIN posting lists are packed on a page, to reduc

From: Heikki Linnakangas <heikki(dot)linnakangas(at)iki(dot)fi>
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: Rewrite the way GIN posting lists are packed on a page, to reduc
Date: 2014-03-31 12:35:32
Message-ID: E1WUbR6-0002fJ-FU@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Rewrite the way GIN posting lists are packed on a page, to reduce WAL volume.

Inserting (in retail) into the new 9.4 format GIN posting tree created much
larger WAL records than in 9.3. The previous strategy to WAL logging was
basically to log the whole page on each change, with the exception of
completely unmodified segments up to the first modified one. That was not
too bad when appending to the end of the page, as only the last segment had
to be WAL-logged, but per Fujii Masao's testing, even that produced 2x the
WAL volume that 9.3 did.

The new strategy is to keep track of changes to the posting lists in a more
fine-grained fashion, and also make the repacking" code smarter to avoid
decoding and re-encoding segments unnecessarily.

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/14d02f0bb352d70d50106e153aca4af9c4b0b842

Modified Files
--------------
src/backend/access/gin/gindatapage.c | 629 +++++++++++++++++++++----------
src/backend/access/gin/ginpostinglist.c | 3 +-
src/backend/access/gin/ginxlog.c | 161 +++++++-
src/backend/access/rmgrdesc/gindesc.c | 65 +++-
src/include/access/gin_private.h | 35 +-
5 files changed, 666 insertions(+), 227 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Robert Haas 2014-03-31 17:03:41 pgsql: Fix thinko in logical decoding code.
Previous Message Heikki Linnakangas 2014-03-31 10:25:21 Re: pgsql: Allow opclasses to provide tri-valued GIN consistent functions.