Re: User concurrency thresholding: where do I look?

Awww Josh,

I was just enjoying the chat on the picket fence! :-)

Anyway the workload is mixed (reads,writes) with simple to medium
queries. The workload is known to scale well. But inorder to provide
substantial input I am still trying to eliminate things that can
bottleneck. Currently I have eliminated CPU (plenty free) , RAM
(memory is 48GB RAM in this server for a 32-bit postgresql instance),
IO Storage (used the free ram to do /tmp database to eliminate IO) and
am still trying to eliminate any network bottlenecks to say for sure we
have a problem in PostgreSQL. But yes till that final thing is confirmed
(network which can very well be the case) it could be a problem
somewhere else. However the thing that worries me is more of the big
drop instead of remaining constant out there..

Anyway more on this within a day or two once I add more network nics
between the systems to eliminate network problems (even though stats
dont show them as problems right now) and also reduce malloc lock
penalties if any.

As for other questions:

max_locks_per_transactions is set to default (10 I believe) increasing
it still seems to degrade overall throughput number.

max_connections is set to 1500 for now till I get decent scaling till
1400-1500 users.

There are no hard failures reported anywhere. Log min durations does
show that queries are now slowing down and taking longer.

OS is not swapping and also eliminated IO by putting the whole database
on /tmp

So while I finish adding more network connections between the two
systems (need to get cards) do enjoy the following URL :-)

http://www.spec.org/jAppServer2004/results/res2007q3/jAppServer2004-20070703-00073.html

Of course all postgresql.conf still remains from the old test so no
flames on that one again :-)

Regards,
Jignesh

Josh Berkus wrote:
> Tom,
>
>
>> Well, if the load is a lot of short writing transactions then you'd
>> expect the throughput to depend on how fast stuff can be pushed down to
>> WAL. What have you got wal_buffers set to? Are you using a commit
>> delay? What's the I/O system anyway (any BB write cache on the WAL
>> disk?) and what wal sync method are you using?
>>
>
> You know, I think Jignesh needs to me on this list so I can stop relaying
> questions on a workload I didn't design. Let me get him.
>
>