Re: Re: How to track number of connections and hosts to Postgres cluster

On Thu, Sep 1, 2011 at 11:46 PM, Venkat Balaji <venkat(dot)balaji(at)verse(dot)in> wrote:
> Hi Scott,
> Log generation rate -
> 500MB size of log file is generated within minimum 3 mins to maximum of 20
> mins depending on the database behavior.
> I did not understand the "fsync" stuff you mentioned. Please help me know
> how would fsync is related to log generation or logging host IPs in the log

So you're generating logs at a rate of about 166MB a minute or 2.7MB/s
Seagates from the early 90s are faster than that. Are you logging
more than just connections and disconnections? If you log just those
what's the rate?

fsync is when the OS says to write to disk and the disk confirms the
write is complete. It probably doesn't matter here whether the file
system is using a journaling method that's real safe or not, and you
can go to something like ext2 where there's no journaling and probably
do fine on a dedicated SATA drive or pair if you want them redundant.

The real issue then is what to do with old log files. Right now
you're creating them at 10G an hour, or 240G a day. So you'll need
some cron job to go in and delete the old ones. Still with a 1TB
drive it'll take about 4 days to fill up, so it's not like you're
gonna run out of space in a few minutes or anything.

Since log files are pretty much written sequentially they don't need
the fastest drives ever made. Most modern 7200RPM 3.5" SATA drives
can write at least at 50 or 60 MB/s on their slowest portions. Just
rotate them hourly or daily or whatever and process them and delete
them.