Re: improving concurrent transactin commit rate

On Wed, Mar 25, 2009 at 05:56:02PM +0000, Sam Mason wrote:
> On Wed, Mar 25, 2009 at 12:01:57PM -0500, Kenneth Marshall wrote:
> > On Wed, Mar 25, 2009 at 03:58:06PM +0000, Sam Mason wrote:
> > > #!/bin/bash
> > > nclients=$1
> > > ittrs=$2
> > > function gensql {
> > > echo "INSERT INTO bm (c,v) VALUES ('$1','0');"
> > > for (( i = 1; i < $ittrs; i++ )); do
> > > echo "UPDATE bm SET v = '$i' WHERE c = '$1';"
> > > done
> > > echo "DELETE FROM bm WHERE c = '$1';"
> > > }
> > > for (( c = 0; c < $nclients; c++)); do
> > > gensql $c | psql -Xq -f - &
> > > done
> > > for (( c = 0; c < $nclients; c++)); do
> > > wait
> > > done
> >
> > Are you sure that you are able to actually drive the load at the
> > high end of the test regime? You may need to use multiple clients
> > to simulate the load effectively.
>
> Notice that the code is putting things into the background and then
> waiting for them to finish so there will be multiple clients. Or maybe
> I'm misunderstanding what you mean.
>
> I've just tried modifying the code to write the generated SQL out to
> a set of files first and this speeds things up by about 6% (the 48
> client case goes from taking ~42 seconds to ~39 seconds) indicating that
> everything is probably OK with the test harness. Also note that this 6%
> improvement will be linear and across the board and hence should just
> appear as slightly reduced performance for my system. As I'm not really
> interested in absolute performance and more in how the system scales as
> load increases this will negate this effect even further.
>
> --
> Sam http://samason.me.uk/
>

I did notice how your test harness was designed. It just seemed that
the actual process contention on your load generation system will actually
bottle-neck as the number of clients increases and that may be the cause
of your fall-off, or a contributor. You could test it by generating the
load from independent boxes and see how the perfomance falls-off as you
add additional load clients+boxes.

My two cents,
Ken