Re: Bitmap Indexes: request for feedback

Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
> On Wed, 2008-10-22 at 08:31 -0400, Tom Lane wrote:
>> No, that was merely one component of the problem. The APIs for
>> tidbitmaps need revision too. You can't "stream" a bitmap yet.

> Please explain further.
> Which calls? Why do we need to stream them?

Well, I guess we don't absolutely *have* to --- we could insist that a
bitmap scan on a bitmap index proceed by first sucking the whole bitmap
into memory, the same as is done for other index types. It seems pretty
silly though, especially for the expected use-case of low cardinality;
the bitmaps would get big.

The idea that was being kicked around was to make it possible for
TIDBitmap to be an alias representing an indexscan in progress. So
you'd pull an index page or so's worth of TIDs from the index, hand
them back to nodeBitmapHeapscan.c to read those tuples, repeat till
done. With judicious use of a few "method" function pointers,
nodeBitmapHeapscan wouldn't need to know whether it was doing this or
using an in-memory bitmap. ANDing and ORing of bitmaps could be made
to work streamably too.

As Greg said, the major complaint against the original patch was that
they'd made it do this (interleave the heap and index access) by hack
slash and burn instead of extending the existing APIs appropriately.

regards, tom lane