Re: pgsql: Compress GIN posting lists, for smaller index size.

From: Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>
To: Andres Freund <andres(at)2ndquadrant(dot)com>
Cc: <pgsql-committers(at)postgresql(dot)org>
Subject: Re: pgsql: Compress GIN posting lists, for smaller index size.
Date: 2014-09-02 11:26:51
Message-ID: 5405A97B.6050907@vmware.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers pgsql-hackers

On 09/02/2014 02:11 PM, Andres Freund wrote:
> Hi,
> On 2014-01-22 17:28:48 +0000, Heikki Linnakangas wrote:
>> Compress GIN posting lists, for smaller index size.
>>
>> GIN posting lists are now encoded using varbyte-encoding, which allows them
>> to fit in much smaller space than the straight ItemPointer array format used
>> before. The new encoding is used for both the lists stored in-line in entry
>> tree items, and in posting tree leaf pages.
>>
>> To maintain backwards-compatibility and keep pg_upgrade working, the code
>> can still read old-style pages and tuples. Posting tree leaf pages in the
>> new format are flagged with GIN_COMPRESSED flag, to distinguish old and new
>> format pages. Likewise, entry tree tuples in the new format have a
>> GIN_ITUP_COMPRESSED flag set in a bit that was previously unused.
>
> A new version of clang complains:
>
> /home/andres/src/postgresql/src/backend/access/gin/ginvacuum.c:512:34: warning: signed shift result (0x80000000) sets the sign bit of the
> shift expression's type ('int') and becomes negative [-Wshift-sign-overflow]
> uncompressed = (ItemPointer) GinGetPosting(itup);
> ^~~~~~~~~~~~~~~~~~~
> /home/andres/src/postgresql/src/include/access/gin_private.h:226:59: note: expanded from macro 'GinGetPosting'
> #define GinGetPosting(itup) ((Pointer) ((char*)(itup) + GinGetPostingOffset(itup)))
> ^~~~~~~~~~~~~~~~~~~~~~~~~
> /home/andres/src/postgresql/src/include/access/gin_private.h:224:85: note: expanded from macro 'GinGetPostingOffset'
> #define GinGetPostingOffset(itup) (GinItemPointerGetBlockNumber(&(itup)->t_tid) & (~GIN_ITUP_COMPRESSED))
> ^~~~~~~~~~~~~~~~~~~
> /home/andres/src/postgresql/src/include/access/gin_private.h:223:33: note: expanded from macro 'GIN_ITUP_COMPRESSED'
> #define GIN_ITUP_COMPRESSED (1 << 31)
>
> As far as I understand the code it should rather be
> #define GIN_ITUP_COMPRESSED (1U << 31)
>
> Is that right?

Yep. Fixed, thanks.

- Heikki

In response to

Browse pgsql-committers by date

  From Date Subject
Next Message Heikki Linnakangas 2014-09-02 12:20:38 pgsql: Refactor per-page logic common to all redo routines to a new fun
Previous Message Heikki Linnakangas 2014-09-02 11:26:30 pgsql: Silence warning on new versions of clang.

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2014-09-02 11:30:50 Re: why after increase the hash table partitions, TPMC decrease
Previous Message Andres Freund 2014-09-02 11:20:46 Re: add line number as prompt option to psql