From: | Sachin Kotwal <kotsachin(at)gmail(dot)com> |
---|---|
To: | Fabien COELHO <coelho(at)cri(dot)ensmp(dot)fr> |
Cc: | PostgreSQL Developers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: pgbench unable to scale beyond 100 concurrent connections |
Date: | 2016-06-29 12:36:43 |
Message-ID: | CA+N_YAewy+cWWZiiC2KhQo5Hgz870Xw8RQBr-cyjBdQVMLgcYA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi Fabien,
Sorry for very short report.
I feel pgbench is not so complex tool.
Please see below answers to your questions.
On Wed, Jun 29, 2016 at 5:07 PM, Fabien COELHO <coelho(at)cri(dot)ensmp(dot)fr> wrote:
>
> Hello Sachin,
>
> Your report is very imprecise so it is hard to tell anything.
>
> What version of client and server are you running?
I am testing it with 9.6-beta1 binaries. For server and client it is same.
I am using pgbench on top of postgres_fdw.
> On what hardware ? (200 connections => 200 active postgres processes, how
> many processes per core are you expecting to run?
I am running in small virtual box machine. with 1GB RAM and 2 cores.
I think there should not be problem with 200 processes on 2 core machines.
I tested same number of concurrent connections on same machine with
sysbench it is working fine.
I am not sure what is difference between pgbench and sysbench, they might
be process based or thread based.
But I can say if I can create 200+ concurrent connection with sysbench , i
should be able create same with pgbench.
Thoughts?
> the recommanded value is about 2 connections per physical core...)
I think 2 connections per core is very small value . for 200 i need atleast
100 core machine , which is not good.
What precise command is started?
> How to you know it "comes down to 100 connections"?
I put watch on live connections to database.
something like : watch -n 1 'ps -ef | grep postgres | grep 192.168.56.101 |
wc -l'
NOTE: grep cxommand may change as per environment.
Also user below query to see active connection.
# select count(*) from pg_stat_activity;
> Are there error messages from pgbench or postgresql?
>
> postgresql does not give any error.
pgbench says:
client 36 aborted in state 2: ERROR: could not connect to server "server_1"
DETAIL: FATAL: sorry, too many clients already
> My random guess would be that you start too many connections with only one
> thread client side and/or on a too small hardware client or server-side for
> the expected scale, so given the load and latency some connections just
> never get to do anything?
>
> This may be reason but it should be able to maintain idle connection for
that time if never get to do anything.
> Maybe try with "-j 20" so that there are not too many connections per
> pgbench thread?
>
> I do not have such good hardware for now.
I feel pgbench should be able to perform well on small hardware.
Feel free to ask any question regarding setup.
> I am testing pgbench with more than 100 connections. also set
>> max_connection in postgresql.conf more than 100.
>>
>> Initially pgbench tries to scale nearby 150 but later it come down to 100
>> connections and stable there.
>>
>> It this limitation of pgbench? or bug? or i am doing it wrong way?
>>
>> ---
>> I tested it with max_connection = 200 in postgresql.conf
>> and pgbench witn -c 180/190/200
>>
>
> Please reply.
>>
>
> Please send precise information instead of expecting people to guess...
>
> --
> Fabien
>
--
Thanks and Regards,
Sachin Kotwal
From | Date | Subject | |
---|---|---|---|
Next Message | M Enrique | 2016-06-29 12:57:56 | Re: Gin index on array of uuid |
Previous Message | David G. Johnston | 2016-06-29 12:30:01 | Re: Strange behavior of some volatile function like random(), nextval() |