From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | "Jim C(dot) Nasby" <jnasby(at)pervasive(dot)com> |
Cc: | Mark Lewis <mark(dot)lewis(at)mir3(dot)com>, Jeffrey Tenny <jeffrey(dot)tenny(at)comcast(dot)net>, pgsql-performance(at)postgresql(dot)org |
Subject: | Re: performance question (something to do w/ |
Date: | 2006-05-08 17:59:56 |
Message-ID: | 28288.1147111196@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers pgsql-performance |
"Jim C. Nasby" <jnasby(at)pervasive(dot)com> writes:
> On Mon, May 08, 2006 at 10:42:21AM -0700, Mark Lewis wrote:
>> Doing a SELECT with a large list of variables inside an IN runs slowly
>> on every database we've tested. We've tested mostly in Oracle and
>> PostgreSQL, and both get very slow very quickly (actually Oracle refuses
>> to process the query at all after it gets too many bind parameters).
>>
>> In our case, we have a (potentially quite large) set of external values
>> that we want to look up in the database. We originally thought that
>> doing a single select with a large IN clause was the way to go, but then
>> we did some performance analysis on the optimal batch size (number of
>> items to include per IN clause), and discovered that for most databases,
>> the optimal batch size was 1. For PostgreSQL I think it was 2.
>>
>> The moral of the story is that you're probably better off running a
>> bunch of small selects than in trying to optimize things with one
>> gargantuan select.
> Ever experiment with loading the parameters into a temp table and
> joining to that?
Also, it might be worth re-testing that conclusion with PG CVS tip
(or 8.2 when it comes out). The reimplementation of IN as = ANY that
I did a couple months ago might well change the results.
regards, tom lane
From | Date | Subject | |
---|---|---|---|
Next Message | Kenneth Marshall | 2006-05-08 18:01:36 | Re: performance question (something to do w/ |
Previous Message | Magnus Hagander | 2006-05-08 17:59:05 | Re: Pragma linking? |
From | Date | Subject | |
---|---|---|---|
Next Message | Kenneth Marshall | 2006-05-08 18:01:36 | Re: performance question (something to do w/ |
Previous Message | Jim C. Nasby | 2006-05-08 17:50:13 | Re: performance question (something to do w/ |