Filtering the results of UNION ALL vs filtering the separate queries

From: Stefan Weiss <weiss(at)foo(dot)at>
To: pgsql-general(at)postgresql(dot)org
Subject: Filtering the results of UNION ALL vs filtering the separate queries
Date: 2016-04-27 13:23:21
Message-ID: 5720BD49.3040108@foo.at
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi.

I'm searching in a medium-sized table (135k rows, 29 columns). Some of the
records point to other (parent) records, whose data values have to be used
for filtering as well as for joins, instead of the record's own fields.

Grouping the different types of records into "subset" views, the query looks
like this:

SELECT ... FROM subset1 JOIN (tables...) WHERE (filters...)
UNION ALL
SELECT ... FROM subset2 JOIN (tables...) WHERE (filters...)
UNION ALL
SELECT ... FROM subset3 JOIN (tables...) WHERE (filters...)

"(tables...)" and "(filters...)" are exactly the same for all selects. Since
the final form of this query may have up to 20 WHERE filters and 14 joins to
other tables, I tried to eliminate this duplication and apply the filters
and joins only to the result of the unfiltered union:

SELECT combined.*
FROM (
SELECT * FROM subset1
UNION ALL
SELECT * FROM subset2
UNION ALL
SELECT * FROM subset3
) combined
JOIN (tables...)
WHERE (filters...)

This gives the same result, but increases the execution time from 9ms to
500ms in the simplest case (1 join, 1 filter). The views are not the cause
of the slowdown; I've also tried this with the view SQL inlined. The main
reason seems to be that the second example needs three seq scans of the
underlying table, while the fast query can make use of indexes (from the
join, I assume).

Is there a way to have the performance of the first query example while
avoiding the duplication of filters and joins?

(I had originally posted a similar question on dba.stackexchange.com [1].
That question has more details, including query plans, but I realize now
that it probably wasn't a good fit for that site.)

Thanks in advance,
Stefan

[1] http://dba.stackexchange.com/questions/136653/

Browse pgsql-general by date

  From Date Subject
Next Message Alex Ignatov 2016-04-27 13:53:36 Re: Does this perf output seem 'normal'?
Previous Message Tim van der Linden 2016-04-27 13:21:52 Re: Slow join over three tables