From: | Luke Lonergan <llonergan(at)greenplum(dot)com> |
---|---|
To: | Greg Smith <gsmith(at)gregsmith(dot)com>, <pgsql-performance(at)postgresql(dot)org> |
Subject: | Re: Benchmark Data requested --- pgloader CE design ideas |
Date: | 2008-02-06 16:17:42 |
Message-ID: | C3CF1DA6.53067%llonergan@greenplum.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-performance |
Hi Greg,
On 2/6/08 7:56 AM, "Greg Smith" <gsmith(at)gregsmith(dot)com> wrote:
> If I'm loading a TB file, odds are good I can split that into 4 or more
> vertical pieces (say rows 1-25%, 25-50%, 50-75%, 75-100%), start 4 loaders
> at once, and get way more than 1 disk worth of throughput reading. You
> have to play with the exact number because if you push the split too far
> you introduce seek slowdown instead of improvements, but that's the basic
> design I'd like to see one day. It's not parallel loading that's useful
> for the cases I'm thinking about until something like this comes around.
Just load 4 relfiles. You have to be able to handle partial relfiles, which
changes the storage mgmt a bit, but the benefits are easier to achieve.
- Luke
From | Date | Subject | |
---|---|---|---|
Next Message | Jignesh K. Shah | 2008-02-06 16:34:14 | Re: Benchmark Data requested --- pgloader CE design ideas |
Previous Message | Tom Lane | 2008-02-06 16:00:26 | Re: Optimizer : query rewrite and execution plan ? |