pgsql: When creating a large hash index, pre-sort the index entries by

From: tgl(at)postgresql(dot)org (Tom Lane)
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: When creating a large hash index, pre-sort the index entries by
Date: 2008-03-16 23:15:08
Message-ID: 20080316231508.EBA0C7558DC@cvs.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Log Message:
-----------
When creating a large hash index, pre-sort the index entries by estimated
bucket number, so as to ensure locality of access to the index during the
insertion step. Without this, building an index significantly larger than
available RAM takes a very long time because of thrashing. On the other
hand, sorting is just useless overhead when the index does fit in RAM.
We choose to sort when the initial index size exceeds effective_cache_size.

This is a revised version of work by Tom Raney and Shreya Bhargava.

Modified Files:
--------------
pgsql/src/backend/access/hash:
Makefile (r1.14 -> r1.15)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/hash/Makefile?r1=1.14&r2=1.15)
hash.c (r1.99 -> r1.100)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/hash/hash.c?r1=1.99&r2=1.100)
hashpage.c (r1.73 -> r1.74)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/hash/hashpage.c?r1=1.73&r2=1.74)
pgsql/src/backend/access/nbtree:
nbtsort.c (r1.114 -> r1.115)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/nbtree/nbtsort.c?r1=1.114&r2=1.115)
pgsql/src/backend/utils/sort:
tuplesort.c (r1.81 -> r1.82)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/sort/tuplesort.c?r1=1.81&r2=1.82)
pgsql/src/include/access:
hash.h (r1.85 -> r1.86)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/access/hash.h?r1=1.85&r2=1.86)
pgsql/src/include/utils:
tuplesort.h (r1.28 -> r1.29)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/utils/tuplesort.h?r1=1.28&r2=1.29)

Added Files:
-----------
pgsql/src/backend/access/hash:
hashsort.c (r1.1)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/hash/hashsort.c?rev=1.1&content-type=text/x-cvsweb-markup)

Browse pgsql-committers by date

  From Date Subject
Next Message User Rbt 2008-03-16 23:30:09 autodoc - autodoc: 1.31 actual
Previous Message Alvaro Herrera 2008-03-16 19:47:34 pgsql: Modify interactions between sinval.c and sinvaladt.c.