Re: Benchmarking a large server

2011/5/10 Greg Smith <greg(at)2ndquadrant(dot)com>:
> On 05/09/2011 11:13 PM, Shaun Thomas wrote:
>>
>> Take a look at /proc/sys/vm/dirty_ratio and
>> /proc/sys/vm/dirty_background_ratio if you have an older Linux system, or
>> /proc/sys/vm/dirty_bytes, and /proc/sys/vm/dirty_background_bytes with a
>> newer one.
>> On older systems for instance, those are set to 40 and 20 respectively
>> (recent kernels cut these in half).
>
> 1/4 actually; 10% and 5% starting in kernel 2.6.22.  The main sources of
> this on otherwise new servers I see are RedHat Linux RHEL5 systems  running
> 2.6.18.  But as you say, even the lower defaults of the newer kernels can be
> way too much on a system with lots of RAM.

one can experiment writeback storm with this script from Chris Mason,
under GPLv2:
http://oss.oracle.com/~mason/fsync-tester.c

You need to tweak it a bit, AFAIR, this #define SIZE (32768*32) must
be reduced to be equal to 8kb blocks if you want similar to pg write
pattern.

The script does a big file, many small fsync, writing on both. Please,
see http://www.spinics.net/lists/linux-ext4/msg24308.html

It is used as a torture program by some linuxfs-hackers and may be
useful for the OP on his large server to validate hardware and kernel.

>
> The main downside I've seen of addressing this by using a kernel with
> dirty_bytes and dirty_background_bytes is that VACUUM can slow down
> considerably.  It really relies on the filesystem having a lot of write
> cache to perform well.  In many cases people are happy with VACUUM
> throttling if it means nasty I/O spikes go away, but the trade-offs here are
> still painful at times.
>
> --
> Greg Smith   2ndQuadrant US    greg(at)2ndQuadrant(dot)com   Baltimore, MD
> PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us
> "PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>

--
Cédric Villemain               2ndQuadrant
http://2ndQuadrant.fr/     PostgreSQL : Expertise, Formation et Support