Re: How to tame a gigantic (100+ lines) query in a web app?

From: Rob Sargent <robjsargent(at)gmail(dot)com>
To: pgsql-general(at)postgresql(dot)org
Cc: "W(dot) Matthew Wilson" <matt(at)tplus1(dot)com>
Subject: Re: How to tame a gigantic (100+ lines) query in a web app?
Date: 2011-08-15 02:59:52
Message-ID: 4E488BA8.1020309@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

W. Matthew Wilson wrote:
> I'm sure I'm not the first person to end up with a gigantic query that
> does lots of left joins and subselects.
>
> It seems to work, but I would love to break it up into smaller chunks.
>
> I'm thinking about rewriting the query to make several temporary
> tables that are dropped on commit, and then joining them at the end.
>
> I can't just use views for everything because I use parameters passed
> in from the web app. I am using a few views where I can.
>
> Is there anything dangerous about making temporary tables in this way?
> I started two transactions simultaneously and they were both able to
> make their own temporary tables.
>
> More generally, how to tame this big ol' query?
>
> The temporary tables mean I'm only pulling data from the database one
> time. ORMs often pull data from one query and then use that data to
> write the next query. This seems slow to me.
>
> Matt
>
>
>
I'm a big fan of the temp table plan, even though they are the
epitome of the very thing of which you indict ORMs! And you might be
wrong on that thesis but who cares, you're in hand-craft mode. My very
biased take is that you will at worst break even performance-wise, and
probably come out on top, especially if a left join goes through an
intersection record.

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Ondrej Ivanič 2011-08-15 05:26:00 Postgres on SSD
Previous Message Ondrej Ivanič 2011-08-15 00:19:10 Re: Where to start, graphs and routing.