Re: queries with lots of UNIONed relations

From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Andy Colson <andy(at)squeakycode(dot)net>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Jon Nelson <jnelson+pgsql(at)jamponi(dot)net>, pgsql-performance(at)postgresql(dot)org
Subject: Re: queries with lots of UNIONed relations
Date: 2011-01-13 22:49:15
Message-ID: AANLkTikg+DmHbRrxdQbeX3B5PyXD0rRziT0QowQ-_LnM@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

On Thu, Jan 13, 2011 at 5:47 PM, Andy Colson <andy(at)squeakycode(dot)net> wrote:
>>>> I don't believe there is any case where hashing each individual relation
>>>> is a win compared to hashing them all together.  If the optimizer were
>>>> smart enough to be considering the situation as a whole, it would always
>>>> do the latter.
>>>
>>> You might be right, but I'm not sure.  Suppose that there are 100
>>> inheritance children, and each has 10,000 distinct values, but none of
>>> them are common between the tables.  In that situation, de-duplicating
>>> each individual table requires a hash table that can hold 10,000
>>> entries.  But deduplicating everything at once requires a hash table
>>> that can hold 1,000,000 entries.
>>>
>>> Or am I all wet?
>>
>> Yeah, I'm all wet, because you'd still have to re-de-duplicate at the
>> end.  But then why did the OP get a speedup?  *scratches head*
>
> Because it all fix it memory and didnt swap to disk?

Doesn't make sense. The re-de-duplication at the end should use the
same amount of memory regardless of whether the individual relations
have already been de-duplicated.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

In response to

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Andy Colson 2011-01-13 22:52:10 Re: queries with lots of UNIONed relations
Previous Message Andy Colson 2011-01-13 22:47:31 Re: queries with lots of UNIONed relations