Re: checkpointer continuous flushing

>> Is it possible to run tests with distinct table spaces on those many disks?
>
> Nope, that'd require reconfiguring the system (and then back), and I don't
> have access to that system (just SSH).

Ok.

> Also, I don't quite see what would that tell us?

Currently the flushing context is shared between table space, but I think
that it should be per table space. My tests did not manage to convince
Andres, so getting some more figures would be great. That will be another
time!

>> I would have suggested using the --latency-limit option to filter out
>> very slow queries, otherwise if the system is stuck it may catch up
>> later, but then this is not representative of "sustainable" performance.
>>
>> When pgbench is running under a target rate, in both runs the
>> transaction distribution is expected to be the same, around 5000 tps,
>> and the green run looks pretty ok with respect to that. The magenta one
>> shows that about 25% of the time, things are not good at all, and the
>> higher figures just show the catching up, which is not really
>> interesting if you asked for a web page and it is finally delivered 1
>> minutes later.
>
> Maybe. But that'd only increase the stress on the system, possibly causing
> more issues, no? And the magenta line is the old code, thus it would only
> increase the improvement of the new code.

Yes and no. I agree that it stresses the system a little more, but the
fact that you have 5000 tps in the end does not show that you can really
sustain 5000 tps with reasonnable latency. I find this later information
more interesting than knowing that you can get 5000 tps on average,
thanks to some catching up. Moreover the non throttled runs already shown
that the system could do 8000 tps, so the bandwidth is already there.

> Notice the max latency is in microseconds (as logged by pgbench), so
> according to the "max latency" charts the latencies are below 10 seconds
> (old) and 1 second (new) about 99% of the time.

AFAICS, the max latency is aggregated by second, but then it does not say
much about the distribution of individuals latencies in the interval, that
is whether they were all close to the max or not, Having the same chart
with median or average might help. Also, with the stddev chart, the
percent do not correspond with the latency one, so it may be that the
latency is high but the stddev is low, i.e. all transactions are equally
bad on the interval, or not.

So I must admit that I'm not clear at all how to interpret the max latency
& stddev charts you provided.

> So I don't think this would make any measurable difference in practice.

I think that it may show that 25% of the time the system could not match
the target tps, even if it can handle much more on average, so the tps
achieved when discarding late transactions would be under 4000 tps.

--
Fabien.