bitmap indexes (Was: [HACKERS] Getting a move on for 8.2 beta)

On 9/2/06, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Gavin Sherry <swm(at)linuxworld(dot)com(dot)au> writes:
> > On Fri, 1 Sep 2006, Tom Lane wrote:
> >> My feeling is that we ought to bounce bitmap indexes and updatable views
> >> as not being ready, accept all the contrib stuff, and try to get the
> >> other items done in time for a beta at, say, the end of next week.
>
> > For what it's worth, Jie and I hope to have finished the bitmap streaming
> > this weekend which is the only outstanding issue with the code as it
> > currently stands.
>
> AFAIR, you told me "it'll be done this weekend" last week. And the week
> before.
>
> In any case, it's going to be a large complicated patch, and if I spend
> all next week reviewing it then I won't make any progress on the other
> items on my to-do list. At some point we have to say "this has slipped
> too far, we have to hold it over for 8.3".

So it seems bitmap indexes won't make it into 8.2.

I've been silently expecting them in pg since last year (from the
talks before Tom introduced bitmap scans/and-ing/or-ing in the
executor) and I'd like to advocate a bit on these as I feel they don't
get enough traction.

Being a Business Intelligence / Data Warehouse shop, our routine is to
use bulk loading, partitions, materialized views and bitmap indexes.
and in the rdbms we're mostly using, the latter do make a huge
difference in star schema query performance. Greenplum also showed
some very encouraging numbers in their april whitepaper on the topic.

There's a great animation on the market right now for all things 'OSS
BI' : BI suites, reporting, ETL, OLAP as well as DW appliances
(Greenplum, DATAllegro, Netezza, ...).
The case of DW appliances is interesting, 2 out of 3 are based on
PostgreSQL (the 3rd is Ingres) which clearly demonstrates the quality
and enterprise nature of pg (not mentioning EnterpriseDB on the OLTP
front).

One space is not really occupied yet: the OSS BI database of choice. I
see 3 contenders : MySQL, PostgreSQL and Bizgres.
- MySQL are actively marketing (and also coding) to take that place.
- PostgreSQL has most features (albeit a bit unpolished) but bitmap
indexes and maybe the will to occupy that space.
- there is also Bizgres but I think it is more perceived as a bridge
between pg and Bizgres MPP and it certainly doesn't have the name
recognition PostgreSQL has, neither the exposure.

I'm also looking for the most appropriate database to use for Mondrian
- a standard-compliant ROLAP system, see
http://mondrian.sourceforge.net - and there pg with bitmap indexes
would be ideal.

My take is that it would be a clever move to position PostgreSQL in
the BI space and now seems to be the best time.

I had planned to do some performance tests on bitmap indexes as soon
as they hit cvs and want to use the tests results to promote pg in the
data warehouse field.

So maybe it's a missed opportunity for 8.2 but I'd like -hackers to
believe that we *are* indeed waiting for on-disk bitmap indexes.

my 2 cents

francois