Re: Mass-Data question

From: Curt Sampson <cjs(at)cynic(dot)net>
To: Gunther Schadow <gunther(at)aurora(dot)regenstrief(dot)org>
Cc: bk <koester(at)x-itec(dot)de>, <pgsql-general(at)postgresql(dot)org>
Subject: Re: Mass-Data question
Date: 2002-04-16 03:25:25
Message-ID: Pine.NEB.4.43.0204161209400.498-100000@angelic.cynic.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

> Boris Kster wrote:
>
> > What should I do if the disk-space is not enough? Is there something
> > to distribute data over several machines and to collect the data with
> > a select statement if required?

> Hmm, interesting. I have similar needs.

As do I. Unfortuantely, I'm not a guru. But I'll be testing out
something like this in the next few weeks if all goes well. I was
planning to do some fairly simple data partitioning. My initial
plan is to drop the data into multiple tables across multiple
servers, partitioned by date, and have a master table indicating
the names of the various tables and the date ranges they cover.
The application will then deal with determining which tables the
query will be spread across, construct and submit the appropriate
queries (eventually in parallel, if I'm getting a lot of queries
crossing multiple tables), and collate the results.

cjs
--
Curt Sampson <cjs(at)cynic(dot)net> +81 90 7737 2974 http://www.netbsd.org
Don't you know, in this new Dark Age, we're all light. --XTC

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Curt Sampson 2002-04-16 03:32:43 Re: psql and passwords
Previous Message Tom Lane 2002-04-16 00:27:40 Re: speeding up subqueries