Re: Proposal of tunable fix for scalability of 8.4

On 03/11/09 18:27, Kevin Grittner wrote:
>>>> "Jignesh K. Shah" <J(dot)K(dot)Shah(at)Sun(dot)COM> wrote:
>>>>
>> Rerunning similar tests on a 64-thread UltraSPARC T2plus based
>> server config
>>
>
>
>> (IO is not a problem... all in RAM .. no disks):
>> Time:Users:Type:TPM: Response Time
>> 60: 100: Medium Throughput: 10552.000 Avg Medium Resp: 0.006
>> 120: 200: Medium Throughput: 22897.000 Avg Medium Resp: 0.006
>> 180: 300: Medium Throughput: 33099.000 Avg Medium Resp: 0.009
>> 240: 400: Medium Throughput: 44692.000 Avg Medium Resp: 0.007
>> 300: 500: Medium Throughput: 56455.000 Avg Medium Resp: 0.007
>> 360: 600: Medium Throughput: 67220.000 Avg Medium Resp: 0.008
>> 420: 700: Medium Throughput: 77592.000 Avg Medium Resp: 0.009
>> 480: 800: Medium Throughput: 87277.000 Avg Medium Resp: 0.011
>> 540: 900: Medium Throughput: 98029.000 Avg Medium Resp: 0.012
>> 600: 1000: Medium Throughput: 102547.000 Avg Medium Resp: 0.023
>>
>
> I'm wondering about the testing methodology. If there is no I/O, I
> wouldn't expect performance to improve after you have all the CPU
> threads busy. (OK, so there might be some brief blocking that would
> make the optimal number of connections somewhat above 64, but 1000???)
>
> What's the bottleneck which allows additional connections to improve
> the throughput? Network latency?
>
> I'm a lot more interested in what's happening between 60 and 180 than
> over 1000, personally. If there was a RAID involved, I'd put it down
> to better use of the numerous spindles, but when it's all in RAM it
> makes no sense.
>
> -Kevin
>

Kevin,

The problem is the CPUs are not all busy there is plenty of idle cycles
since PostgreSQL ends up in situations where they are all waiting for
lockacquires for exclusive.. In cases where there is say one cpu then
waking up one or few waiters is more efficient.. However when you have
64 or 128 or 256 (as in my case), waking up one waiter is inefficient
since only one waiter will be allowed to run while other waiters will
still wake up, spin acquire lock and say.. oh I am still not allowed and
go back to speed..

Testing methology is considering we can get fast storage, can PostgreSQL
still scale to use say 32, 64, 128, 256 cpus... I am just ahead of the
curve of wide spread usage here probably but I want to make sure
PostgreSQL is well tested already for it. And yes I still have plenty of
unused CPU so the goal is to make sure if system can handle it, so can
PostgreSQL.

Regards,
Jignesh