Re: limiting performance impact of wal archiving.

On Tue, Nov 10, 2009 at 3:05 PM, Ivan Voras <ivoras(at)freebsd(dot)org> wrote:
> Laurent Laborde wrote:
>>
>> Hi !
>> We recently had a problem with wal archiving badly impacting the
>> performance of our postgresql master.
>
> Hmmm, do you want to say that copying 16 MB files over the network (and
> presumably you are not doing it absolutely continually - there are pauses
> between log shipping - or you wouldn't be able to use bandwidth limiting) in
> an age when desktop drives easily read 60 MB/s (and besides most of the file
> should be cached by the OS anyway) is a problem for you? Slow hardware?
>
> (or I've misunderstood the problem...)

Desktop drive can easily do 60MB/s in *sequential* read/write.
We use high performance array of 15.000rpm SAS disk on an octocore
32GB and IO is always a problem.

I explain the problem :

This server (doing wal archiving) is the master node of the
over-blog's server farm.
hundreds of GB of data, tens of millions of articles and comments,
millions of user, ...
~250 read/write sql requests per seconds for the master
~500 read sql request per slave.

Awefully random access overload our array at 10MB/s at best.
Of course, when doing sequential read it goes to +250MB/s :)

Waiting for "cheap" memory to be cheap enough to have 512Go of ram per server ;)

We tought about SSD.
But interleaved read/write kill any SSD performance and is not better
than SSD. Just more expensive with an unknown behaviour over age.

--
ker2x
sysadmin & DBA @ http://www.over-blog.com/