From: | "Nigel J(dot) Andrews" <nandrews(at)investsystems(dot)co(dot)uk> |
---|---|
To: | Shridhar Daithankar <shridhar_daithankar(at)persistent(dot)co(dot)in> |
Cc: | pgsql-general <pgsql-general(at)postgresql(dot)org> |
Subject: | Re: Large databases, performance |
Date: | 2002-10-03 12:56:03 |
Message-ID: | Pine.LNX.4.21.0210031353540.26902-100000@ponder.fairway2k.co.uk |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general pgsql-hackers pgsql-performance pgsql-sql |
Shridhar,
It's one hell of a DB you're building. I'm sure I'm not the only one interested
so to satisfy those of us who are nosey: can you say what the application is?
I'm sure we'll all understand if it's not possible for you mention such
information.
--
Nigel J. Andrews
On Thu, 3 Oct 2002, Shridhar Daithankar wrote:
> Hi,
>
> Today we concluded test for database performance. Attached are results and the
> schema, for those who have missed earlier discussion on this.
>
> We have (almost) decided that we will partition the data across machines. The
> theme is, after every some short interval a burst of data will be entered in
> new table in database, indexed and vacuume. The table(s) will be inherited so
> that query on base table will fetch results from all the children. The
> application has to consolidate all the data per node basis. If the database is
> not postgresql, app. has to consolidate data across partitions as well.
>
> Now we need to investigate whether selecting on base table to include children
> would use indexes created on children table.
>
> It's estimated that when entire data is gathered, total number of children
> tables would be around 1K-1.1K across all machines.
>
> This is in point of average rate of data insertion i.e. 5K records/sec and
> total data size, estimated to be 9 billion rows max i.e. estimated database
> size is 900GB. Obviously it's impossible to keep insertion rate on an indexed
> table high as data grows. So partitioning/inheritance looks better approach.
>
> Postgresql is not the final winner as yet. Mysql is in close range. I will keep
> you guys posted about the result.
>
> Let me know about any comments..
>
> Bye
> Shridhar
From | Date | Subject | |
---|---|---|---|
Next Message | Joe Lester | 2002-10-03 13:44:50 | Inquiry From Form [pgsql] |
Previous Message | Charles H. Woloszynski | 2002-10-03 12:54:29 | Re: Large databases, performance |
From | Date | Subject | |
---|---|---|---|
Next Message | Philip Warner | 2002-10-03 13:10:48 | Re: pg_dump and large files - is this a problem? |
Previous Message | Charles H. Woloszynski | 2002-10-03 12:54:29 | Re: Large databases, performance |
From | Date | Subject | |
---|---|---|---|
Next Message | Shridhar Daithankar | 2002-10-03 14:03:30 | Re: Large databases, performance |
Previous Message | Charles H. Woloszynski | 2002-10-03 12:54:29 | Re: Large databases, performance |
From | Date | Subject | |
---|---|---|---|
Next Message | Shridhar Daithankar | 2002-10-03 14:03:30 | Re: Large databases, performance |
Previous Message | Charles H. Woloszynski | 2002-10-03 12:54:29 | Re: Large databases, performance |