Re: Best way to load test a postgresql server

On Wed, 03 Jun 2009 05:29:02 -0400, Dimitri Fontaine
<dfontaine(at)hi-media(dot)com> wrote:
> Last time I used it it was in the context of a web application and to
> compare PostgreSQL against Informix after a migration. So I used the
> HTTP protocol support of the injector.

Tsung seems well suited for that.

> Tsung is based on erlang...you could run 600 to 800 concurrent clients
> from each node.

But each tsung session (virtual user) uses a separate PG connection, and I
need 30k virtual users. I can't imagine 30k PG connections. I could
imagine using pgbouncer in statement pooling mode, but that doesn't
characterize my load well, where different PG connections have different
profiles. I have about 500 connections:

~450 from web servers, often idle, various work loads, no prepared
statements
50 from another client, mostly idle, small set of prepared statements
10 from another client, extremely active, small set of prepared
statements

I know a tsung session doesn't have to exactly mimic a user and I tried to
coerce a tsung session to represent instead a DB client, with loops and
multiple CSV files. I wasn't so successful there, and was nagged by the
assignment of sessions by probability, when I wanted a fixed number
running each session.

I do appreciate the suggestions, and I agree Tsung has lots of nifty
features. I used pgfouine to generate tsung sessions I love the graph
generation but for me it comes down to simulating my DB load so that I can
profile and tune the DB. I am not seeing how to get tsung to fit my case.

Next up I will try JMeter (thanks Shaul Dar for the suggestions).

Regards,
Ken