Problem with large number of index conditions

From: Brian Herlihy <btherl(at)yahoo(dot)com(dot)au>
To: pgsql-bugs(at)postgresql(dot)org
Subject: Problem with large number of index conditions
Date: 2005-08-01 03:38:23
Message-ID: 20050801033823.9135.qmail@web52308.mail.yahoo.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Hi,

I posted this to pgsql-performance a month ago, but have yet to receive a
response, so I am trying this list instead.

I am using PostgreSQL 7.4.7

I have a query looking something like this:

SELECT * FROM tbl WHERE (n = 0::smallint AND (string = 'Hello' OR string =
'Pineapple' OR string = 'Kitten' OR ..... string = 'Potato));

There is an index on (n, string) as well as an index on (string). The casting
of n allows use of the (n, string) index.

The problem is that once the number of 'OR' conditions goes past around 500,
postgresql will revert to doing a sequential scan with a filter. This kills
performance, as the table has 3 million rows. Setting enable_seqscan to OFF
does not affect the query plan.

I have solved the problem by breaking the query into sections, as follows:

SELECT * FROM tbl WHERE (n = 0 AND (string = 'Hello' OR string = 'Pineapple' OR
string = 'Kitten' OR ..... string = 'Piano'))
UNION
SELECT * FROM tbl WHERE (n = 0 AND (string = 'Panic' OR ... OR string =
'Potato));

where each subquery has no more then 200 conditions. Each subquery uses the
index, and then the results are unioned. This takes a few seconds to run,
whereas the sequential scan can take an hour (on a loaded system).

Is there any better solution?

Thanks,
Brian Herlihy

Browse pgsql-bugs by date

  From Date Subject
Next Message Aaron Harsh 2005-08-01 05:02:30 BUG #1800: "unexpected chunk number" during pg_dump
Previous Message Sean Burlington 2005-07-31 16:16:32 Re: BUG #1794: inheritance removes permissions from the parent