From: | Thor Tall <tall_thor(at)yahoo(dot)com> |
---|---|
To: | pgsql-general <pgsql-general(at)postgresql(dot)org> |
Subject: | Slow insert/delete |
Date: | 2006-08-23 14:33:14 |
Message-ID: | 20060823143314.21070.qmail@web50307.mail.yahoo.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
I am in the process of converting an old system to a
new system where I have chosen to use postgres in
stead of a home grow system based on b-trees.
The system receives 2650 message a total of 10Mbytes
of data per 15 minutes this information have to be
store in 4 tables in the database. Some of the
messages are updates to existing data and the result
is:
1038 records in table 1 size per record 282 bytes
2074 records in table 2 size per record 36 bytes
21488 records in table 3 size per record 60 bytes
25676 records in table 4 size per record 42 bytes.
Without indexes this should be something like
2,7Mbytes/15min.
The disk speed is as follows taken from hdparm -Tt
/dev/hda:
Timing buffer-cache reads: 128 MB in 0.89 seconds
=143.82 MB/sec
Timing buffered disk reads: 64 MB in 1.45 seconds =
44.14 MB/sec
My problem is that the postmaster process is using
between 70-90% of the CPU time. (seen using top) and
the total wall time with nothing else running.
It takes 6 minutes to ingest the data with an empty
database and 25 minutes (wall time) if all the data is
already in the database.
The processing is done as follows:
1. Start transaction
2. check if message is in table 1 and if so delete
records from table 1(1 row),2(2 rows),3(30 rows),4(50
rows).
3. split the message and insert the date in table 1(1
row),2(2 rows),3(30 rows),4(50 rows)
4. end transaction
I need some suggestions so that I can ingest the data
faster. with the same computer and disks. We are
running redhat 7.2 on the server and I am using the c
interface to write the data to the server.
Thanks for any help,
Thor.
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
From | Date | Subject | |
---|---|---|---|
Next Message | Don Isgitt | 2006-08-23 14:37:17 | Re: pl/R problem |
Previous Message | Alvaro Herrera | 2006-08-23 14:31:45 | Re: Majordomo drops multi-line Subject: |