Re: Linux: more cores = less concurrency.

On Mon, Apr 11, 2011 at 5:06 PM, Kevin Grittner
<Kevin(dot)Grittner(at)wicourts(dot)gov> wrote:
> Glyn Astill <glynastill(at)yahoo(dot)co(dot)uk> wrote:
>
>> The issue I'm seeing is that 8 real cores outperform 16 real
>> cores, which outperform 32 real cores under high concurrency.
>
> With every benchmark I've done of PostgreSQL, the "knee" in the
> performance graph comes right around ((2 * cores) +
> effective_spindle_count).  With the database fully cached (as I
> believe you mentioned), effective_spindle_count is zero.  If you
> don't use a connection pool to limit active transactions to the
> number from that formula, performance drops off.  The more CPUs you
> have, the sharper the drop after the knee.

I was about to say something similar with some canned advice to use a
connection pooler to control this. However, OP scaling is more or
less topping out at cores / 4...yikes!. Here are my suspicions in
rough order:

1. There is scaling problem in client/network/etc. Trivially
disproved, convert the test to pgbench -f and post results
2. The test is in fact i/o bound. Scaling is going to be
hardware/kernel determined. Can we see iostat/vmstat/top snipped
during test run? Maybe no-op is burning you?
3. Locking/concurrency issue in heavy_seat_function() (source for
that?) how much writing does it do?

Can we see some iobound and cpubound pgbench runs on both servers?

merlin