High inserting by syslog

From: "Valter Douglas Lisbôa Jr(dot)" <douglas(at)trenix(dot)com(dot)br>
To: pgsql-general(at)postgresql(dot)org
Subject: High inserting by syslog
Date: 2008-07-03 15:05:15
Message-ID: 200807031205.15558.douglas@trenix.com.br
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hello all, I have a perl script thats load a entire day squid log to a
postgres table. I run it at midnight by cronjob and turns off the indexes
before do it (turning it on after). The script works fine, but I want to
change this to a diferent approach.

I'd like to insert on the fly the log lines, so long it be generated to have
the data on-line. But the table has some indexes and the load of lines is
about 300.000/day, so the average inserting is 3,48/sec. I think this could
overload the database server (i did not test yet), so if I want to create a
no indexed table to receive the on-line inserting and do a job moving all
lines to the main indexed table at midnight.

My question is, Does exists a better solution, or this tatic is a good way to
do this?

--
Valter Douglas Lisbôa Jr.
Sócio-Diretor
Trenix - IT Solutions
"Nossas Idéias, suas Soluções!"
www.trenix.com.br
contato(at)trenix(dot)com(dot)br
Tel. +55 19 3402.2957
Cel. +55 19 9183.4244

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Gregory Stark 2008-07-03 15:09:23 Re: Switching between terminals
Previous Message Artacus 2008-07-03 15:04:24 Re: Date Formatting for dd/mm/yyyy