Re: benchmark results comparing versions 15.2 and 16

I have two more runs of the benchmark in progress so we will have 3 results
for each of the test cases to confirm that the small regressions are
repeatable.

On Fri, May 5, 2023 at 1:34 PM Andres Freund <andres(at)anarazel(dot)de> wrote:

> One thing that's somewhat odd is that there's very marked changes in l.i0's
> p99 latency for the four clients cases - but whether 15 or 16 are better
> differs between the runs.
>

From the response time sections for l.i0 the [1ms, 4ms) bucket has 99% or
more for all 6 cases.
For example,
https://mdcallag.github.io/reports/23_05_04_ibench.beelink.pg16b.1u.1tno.cached/all.html#l.i0.rt
But yes, the p99 is as you state. I will wade through my test scripts
tomorrow to see how the p99 is computed.

I do wonder if there's something getting scheduled in some of these runs
> increasing latency?
>

Do you mean interference from other processes? Both the big and small
servers run Ubuntu 22.04 server (no X) and there shouldn't be many extra
things, although Google Cloud adds a few extra things that run in the
background.

> Or what we're seeing depends on the time between the start of the server
> and
> the start of the benchmark? It is interesting that the per-second
> throughput
> graph shows a lot of up/down at the end:
>
> https://mdcallag.github.io/reports/23_05_04_ibench.beelink.pg16b.4u.1tyes.1g/tput.l.i0.html
>
> Both 15 and 16 have one very high result at 70s, 15 then has one low, but
> 16
> has two low results.
>

Immediately prior to l.i0 the database directory is wiped and then Postgres
is initialized and started.

The IPS vs time graphs are more interesting for benchmark steps that run
longer. Alas, this can't run too long if the resulting database is to fit
in <= 16G. But that is a problem for another day. The IPS vs time graphs
are not a flat line, but I am not ready to pursue that as problem unless it
shows multi-second write-stalls (fortunately it does not).

--
Mark Callaghan
mdcallag(at)gmail(dot)com