From: | Ericson Smith <eric(at)did-it(dot)com> |
---|---|
To: | Adam Siegel <adam(at)sycamorehq(dot)com> |
Cc: | pgsql-hackers(at)postgresql(dot)org, pgsql-performance(at)postgresql(dot)org, pgsql-general(at)postgresql(dot)org |
Subject: | Re: realtime data inserts |
Date: | 2003-05-10 16:02:18 |
Message-ID: | 3EBD228A.8000901@did-it.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general pgsql-hackers pgsql-performance |
Had the same problem recently...
Format your data like a pg text dump into a file and then...
COPY <tablename> (a,b,c) FROM stdin;
1 2 3
4 5 6
\.
psql <yourdatabase < dumpfile.sql
I've achieved thousands of rows per seconds with this method.
- Ericson Smith
eric(at)did-it(dot)com
http://www.did-it.com
Adam Siegel wrote:
>I have realtime data flowing at a rate of 500, 512 byte packets per second.
>I want to log the info in a database table with two other columns, one for a
>timestamp and one for a name of the packet. The max rate I can achieve is
>350 inserts per second on a sun blade 2000. The inserts are grouped in a
>transaction and I commit every 1200 records. I am storing the binary data
>in a bytea. I am using the libpq conversion function. Not sure if that is
>slowing me down. But I think it is the insert not the conversion.
>
>Any thoughts on how to achive this goal?
>
>
>---------------------------(end of broadcast)---------------------------
>TIP 4: Don't 'kill -9' the postmaster
>
>
From | Date | Subject | |
---|---|---|---|
Next Message | Jim C. Nasby | 2003-05-10 17:08:33 | Re: realtime data inserts |
Previous Message | Tom Lane | 2003-05-10 16:00:02 | Re: realtime data inserts |
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2003-05-10 16:10:08 | Re: Plans for index names unique to a table? |
Previous Message | Tom Lane | 2003-05-10 16:00:02 | Re: realtime data inserts |
From | Date | Subject | |
---|---|---|---|
Next Message | Jim C. Nasby | 2003-05-10 17:08:33 | Re: realtime data inserts |
Previous Message | Tom Lane | 2003-05-10 16:00:02 | Re: realtime data inserts |