Re: Linux: more cores = less concurrency.

When purchasing the intel 7500 series, please make sure to check the hemisphere mode of your memory configuration. There is a HUGE difference in the memory configuration around 50% speed if you don't populate all the memory slots on the controllers properly.

https://globalsp.ts.fujitsu.com/dmsp/docs/wp-nehalem-ex-memory-performance-ww-en.pdf

- John

-----Original Message-----
From: pgsql-performance-owner(at)postgresql(dot)org [mailto:pgsql-performance-owner(at)postgresql(dot)org] On Behalf Of Merlin Moncure
Sent: Tuesday, April 12, 2011 12:14 PM
To: Greg Smith
Cc: Kevin Grittner; david(at)lang(dot)hm; Steve Clark; Glyn Astill; Joshua D. Drake; Scott Marlowe; pgsql-performance(at)postgresql(dot)org
Subject: Re: [PERFORM] Linux: more cores = less concurrency.

On Tue, Apr 12, 2011 at 12:00 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> Kevin Grittner wrote:
>>
>> Glyn Astill <glynastill(at)yahoo(dot)co(dot)uk> wrote:
>>
>>>
>>> Results from Greg Smiths stream_scaling test are here:
>>>
>>> http://www.privatepaste.com/4338aa1196
>>>
>>
>>  Well, that pretty much clinches it.  Your RAM access tops out at 16
>> processors.  It appears that your processors are spending most of
>> their time waiting for and contending for the RAM bus.
>>
>
> I've pulled Glyn's results into
> https://github.com/gregs1104/stream-scaling
> so they're easy to compare against similar processors, his system is
> the one labled 4 X X7550.  I'm hearing this same story from multiple people lately:
>  these 32+ core servers bottleneck on aggregate memory speed with
> running PostgreSQL long before the CPUs are fully utilized.  This
> server is close to maximum memory utilization at 8 cores, and the
> small increase in gross throughput above that doesn't seem to be
> making up for the loss in L1 and L2 thrashing from trying to run more.  
> These systems with many cores can only be used fully if you have a
> program that can work efficiency some of the time with just local CPU
> resources.  That's very rarely the case for a database that's moving
> 8K pages, tuple caches, and other forms of working memory around all the time.
>
>
>> I have gotten machines in where moving a jumper, flipping a DIP
>> switch, or changing BIOS options from the default made a big
>> difference.  I'd be looking at the manuals for my motherboard and
>> BIOS right now to see what options there might be to improve that
>
> I already forwarded Glyn a good article about tuning these Dell BIOSs
> in particular from an interesting blog series others here might like too:
>
> http://bleything.net/articles/postgresql-benchmarking-memory.html
>
> Ben Bleything is doing a very thorough walk-through of server hardware
> validation, and as is often the case he's already found one major
> problem with the vendor config he had to fix to get expected results.

For posterity, since it looks like you guys have nailed this one, I took a look at some of the code off list and I can confirm there is no obvious bottleneck coming from locking type issues. The functions are 'stable' as implemented with no fancy tricks.

merlin

--
Sent via pgsql-performance mailing list (pgsql-performance(at)postgresql(dot)org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
This communication is for informational purposes only. It is not
intended as an offer or solicitation for the purchase or sale of
any financial instrument or as an official confirmation of any
transaction. All market prices, data and other information are not
warranted as to completeness or accuracy and are subject to change
without notice. Any comments or statements made herein do not
necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
and affiliates.

This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and any
attachments are believed to be free of any virus or other defect
that might affect any computer system into which it is received and
opened, it is the responsibility of the recipient to ensure that it
is virus free and no responsibility is accepted by JPMorgan Chase &
Co., its subsidiaries and affiliates, as applicable, for any loss
or damage arising in any way from its use. If you received this
transmission in error, please immediately contact the sender and
destroy the material in its entirety, whether in electronic or hard
copy format. Thank you.

Please refer to http://www.jpmorgan.com/pages/disclosures for
disclosures relating to European legal entities.