Re: pgbench calculates summary numbers a wrong way.

At Fri, 18 Sep 2020 17:28:22 +0900 (JST), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in
> > I have submitted a patch which reworks how things are computed so that
> > performance figures make some/more sense, among other things.
> >
> > Maybe you could have a look at it and tell whether it is an
> > improvement wrt your issue?
>
> Thanks for the pointer, I'm going to look it.

I remember about the topic. It started from an issue of thread/client
connection synchronization. Current it seems to be added overhauling
of time counting code. (I think they are in separate patches).
However, the patch doesn't fix my issues.

If I re-explain my issues in few words:

The first issue is: in printResuts, conn_total_delay is assumed to be
the sum of connection delay of all clients. But actually it is the sum
of connection delay of all *threads*. As the results the connection
delay is underestimated when nclients is larger than nthreads, then
tps(excluding conn establishment) is too-small on that condition.

The second issue is: estimated latency average (without -L) is
calculated as time_include * nclients / <# of completed
txns>. Considering that the latency is the time after connection
establishement until transaction end, the "time_include" should be
time_exclude. As the result the estimated (non -L case) average
latency gets overestimated than the measured latency (with -L case) if
connections takes a long time.

The patch doesn't affect the first issue, but alleviates the error in
the second issue. But still it doesn't handles connection delay
correctly in calculating the value when non -C mode, and doesn't fix
the error when -C (conn per tx) mode at all.

So the two patches are almost orthogonal (but heavily conflicts each
other^^;). Anyway, it's a good opportunity, so I'll take a look on
your patch.

Thanks.

--
Kyotaro Horiguchi
NTT Open Source Software Center