| From: | "Pierre C" <lists(at)peufeu(dot)com> |
|---|---|
| To: | "Scott Marlowe" <scott(dot)marlowe(at)gmail(dot)com>, "Shaun Thomas" <sthomas(at)peak6(dot)com> |
| Cc: | "Craig Ringer" <craig(at)postnewspapers(dot)com(dot)au>, "Aren Cambre" <aren(at)arencambre(dot)com>, pgsql-performance(at)postgresql(dot)org |
| Subject: | Re: Postgres refusing to use >1 core |
| Date: | 2011-05-11 18:10:54 |
| Message-ID: | op.vvbwsgdaeorkce@apollo13 |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-performance |
> This is a problem I encounter constantly wherever I go. Programmer
> selects millions of rows from giant table. Programmer loops through
> results one by one doing some magic on them. Programmer submits queries
> back to the database. Even in batches, that's going to take ages.
Reminds me of a recent question on stackoverflow :
And the answer :
OP was thinking "row-based", with subqueries in the role of "doing some
magicm".
Using a set-based solution with cascading WITH CTEs (and using the
previous CTE as a source in the next one for aggregation) => 100x speedup !
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Jeff Janes | 2011-05-11 19:07:07 | Re: 'Interesting' prepared statement slowdown on large table join |
| Previous Message | Tom Lane | 2011-05-11 17:00:26 | Re: Poor performance when joining against inherited tables |