Re: Performance problems testing with Spamassassin

work_mem = 131072 # min 64, size in KB
shared_buffers = 16000 # min 16, at least max_connections*2, 8KB each
checkpoint_segments = 128 # in logfile segments, min 1, 16MB each
effective_cache_size = 750000 # typically 8KB each
fsync=false # turns forced synchronization on or off

------------------------------------------
On Bizgres (0_7_2) running on a 2GHz Opteron:
------------------------------------------
[llonergan(at)stinger4 bayesBenchmark]$ ./test.sh

real 0m38.348s
user 0m1.422s
sys 0m1.870s

------------------------------------------
On a 2.4GHz AMD64:
------------------------------------------
[llonergan(at)kite15 bayesBenchmark]$ ./test.sh

real 0m35.497s
user 0m2.250s
sys 0m0.470s

Now we turn fsync=true:

------------------------------------------
On a 2.4GHz AMD64:
------------------------------------------
[llonergan(at)kite15 bayesBenchmark]$ ./test.sh

real 2m7.368s
user 0m2.560s
sys 0m0.750s

I guess we see the real culprit here. Anyone surprised it's the WAL?

- Luke

________________________________

From: pgsql-performance-owner(at)postgresql(dot)org on behalf of Andrew McMillan
Sent: Thu 7/28/2005 10:50 PM
To: Matthew Schumacher
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

On Thu, 2005-07-28 at 16:13 -0800, Matthew Schumacher wrote:
>
> Ok, I finally got some test data together so that others can test
> without installing SA.
>
> The schema and test dataset is over at
> http://www.aptalaska.net/~matt.s/bayes/bayesBenchmark.tar.gz
>
> I have a pretty fast machine with a tuned postgres and it takes it about
> 2 minutes 30 seconds to load the test data. Since the test data is the
> bayes information on 616 spam messages than comes out to be about 250ms
> per message. While that is doable, it does add quite a bit of overhead
> to the email system.

On my laptop this takes:

real 1m33.758s
user 0m4.285s
sys 0m1.181s

One interesting effect is the data in bayes_vars has a huge number of
updates and needs vacuum _frequently_. After the run a vacuum full
compacts it down from 461 pages to 1 page.

Regards,
Andrew.

-------------------------------------------------------------------------
Andrew @ Catalyst .Net .NZ Ltd, PO Box 11-053, Manners St, Wellington
WEB: http://catalyst.net.nz/ PHYS: Level 2, 150-154 Willis St
DDI: +64(4)803-2201 MOB: +64(272)DEBIAN OFFICE: +64(4)499-2267
I don't do it for the money.
-- Donald Trump, Art of the Deal

-------------------------------------------------------------------------