GiST Indexes
Introduction
Some of the information here is derived from
the University of California, Berkeley, The GiST Indexing Project web site
and Marcel Kornacker's Thesis,
Access Methods for Next-Generation Database Systems. The GiST
implementation in PostgreSQL is primarily maintained by
Teodor Sigaev and Oleg Bartunov, and there is more information on their website:
GiST stands for Generalized Search Tree. It is a balanced,
tree-structured access method, that acts as a base template in which to implement
arbitrary indexing schemes. B+-trees, R-trees and many other indexing schemes
can be implemented in GiST.
One advantage of GiST is that it allows the development of custom
data types with the appropriate access methods, by
an expert in the domain of the data type, rather than a database expert.
Extensibility
Traditionally, implementing a new access method meant a lot of difficult work.
It was necessary to understand the inner workings of the database, such as the
lock manager and Write-Ahead Log. The GiST interface has
a high level of abstraction, requiring the access method implementor to only
implement the semantics of the data type being accessed. The GiST
layer itself takes care of concurrency, logging and searching the tree structure.
This extensibility should not be confused with the extensibility of the other
standard search trees in terms of the data they can handle.
For example, PostgreSQL supports extensible B+-trees and R-trees. That means
that you can use PostgreSQL to build a B+-tree or R-tree over any data type
you want. But B+-trees only support range predicates (<, =, >), and R-trees
only support n-d range queries (contains, contained, equals).
So if you index, say, an image collection with a PostgreSQL
B+-tree, you can only issue queries such as "is imagex equal to imagey", "is imagex less than
imagey" and "is imagex greater than imagey"? Depending on how you define 'equals',
'less than' and 'greater than' in this context, this may be useful. However, by using
a GiST based index, you could ask questions such as "find all
images of horses" and "find all over-exposed images".
All it takes to get a GiST access method up and running is to
implement four user-defined methods, which define the behavior of keys in the tree. Of course these
methods have to be pretty fancy to support fancy queries, but for all
the standard queries (a la B+-trees, R-trees, etc.) they're relatively straightforward. In
short, the GiST combines extensibility along with
generality, code reuse, and a clean interface.
Implementation
There are seven methods required of the access method to use the GiST interface:
consistent
Given a predicate p on a tree page, and a user query, q,
this method will return false if it is certain that both p and
q cannot be true for a given data item.
union
This method consolidates information in the tree. Given a set of entries, this function
generates a new predicate that is true for all the entries.
compress
Converts the data item into a format suitable for physical storage in the page.
decompress
The reverse of the compress method. Converts the page representation
of the data item into a format that can be manipulated by the database.
penalty
Returns a value indicating the 'cost' of inserting the new entry into a particular
branch of the tree. items will be inserted down the path of least penalty
in the tree.
picksplit
When a page split is necessary, this function decides which entries on the page
are to stay on the old page, and which are to move to the new page.
same
Returns true if two entries are identical, false otherwise.
Limitations
The current implementation of GiST withing PostgreSQL
has two major limitations: GiST access is not concurrent and the GiST
interface doesn't allow the development of certain data types, such as digital trees. (See papers
by Aoki et. al)
Solutions to the first of these problems appear in Marcel Kornacker's Thesis, however these ideas
have not yet been put into practice in the PostgreSQL
implementation.
Examples
To see example implementations of index methods implemented using GiST, examine
the following contrib modules:
btree_gist
B-Tree
cube
Indexing for multi-dimensional cubes
intarray
RD-Tree for one-dimensional array of int4 values
ltree
Indexing for tree-like stuctures
rtree_gist
R-Tree
seg
Storage and indexed access for 'float ranges'
tsearch and tsearch2
Full text indexing