Re: Linux: more cores = less concurrency.

--- On Mon, 11/4/11, Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov> wrote:

> From: Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov>
> Subject: Re: [PERFORM] Linux: more cores = less concurrency.
> To: david(at)lang(dot)hm, "Steve Clark" <sclark(at)netwolves(dot)com>, "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>, "Glyn Astill" <glynastill(at)yahoo(dot)co(dot)uk>
> Cc: "Joshua D. Drake" <jd(at)commandprompt(dot)com>, "Scott Marlowe" <scott(dot)marlowe(at)gmail(dot)com>, pgsql-performance(at)postgresql(dot)org
> Date: Monday, 11 April, 2011, 22:35
> "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
> wrote:
>
> > I don't know why you were hitting the knee sooner than
> I've seen
> > in my benchmarks
>
> If you're compiling your own executable, you might try
> boosting
> LOG2_NUM_LOCK_PARTITIONS (defined in lwlocks.h) to 5 or
> 6.  The
> current value of 4 means that there are 16 partitions to
> spread
> contention for the lightweight locks which protect the
> heavyweight
> locking, and this corresponds to your best throughput
> point.  It
> might be instructive to see what happens when you tweak the
> number
> of partitions.
>

Tried tweeking LOG2_NUM_LOCK_PARTITIONS between 5 and 7. My results took a dive when I changed to 32 partitions, and improved as I increaced to 128, but appeared to be happiest at the default of 16.

> Also, if you can profile PostgreSQL at the sweet spot and
> again at a
> pessimal load, comparing the profiles should give good
> clues about
> the points of contention.
>

Results for the same machine on 8 and 32 cores are here:

http://www.8kb.co.uk/server_benchmarks/dblt_results.csv

Here's the sweet spot for 32 cores, and the 8 core equivalent:

http://www.8kb.co.uk/server_benchmarks/iostat-32cores_32Clients.txt
http://www.8kb.co.uk/server_benchmarks/vmstat-32cores_32Clients.txt

http://www.8kb.co.uk/server_benchmarks/iostat-8cores_32Clients.txt
http://www.8kb.co.uk/server_benchmarks/vmstat-8cores_32Clients.txt

... and at the pessimal load for 32 cores, and the 8 core equivalent:

http://www.8kb.co.uk/server_benchmarks/iostat-32cores_100Clients.txt
http://www.8kb.co.uk/server_benchmarks/vmstat-32cores_100Clients.txt

http://www.8kb.co.uk/server_benchmarks/iostat-8cores_100Clients.txt
http://www.8kb.co.uk/server_benchmarks/vmstat-8cores_100Clients.txt

vmstat shows double the context switches on 32 cores, could this be a factor? Is there anything else I'm missing there?

Cheers
Glyn