Re: pgbench throttling latency limit

Hello Heikki,

> I find the definition of the latency limit a bit strange. It's a limit on how
> late a transaction can *start* compared to it's scheduled starting time, not
> how long a query is allowed to last.

Yes. This is what can be done easily with pgbench under throttling. Note
that if transactions take long it is recorded (average, stddev...) so it
appears elsewhere.

> How do figure out what it should be set to?

It is really just a simple tool to measure unresponsiveness under
throttling, which I'm testing and complaining about in another thread.

The underlying model I have in mind would some timeout from an
application, say a web server, or a pooling process which is handling a
queue of requests...

Now, if I describe that as "lag limit" instead if "latency limit", maybe
it is clearer and better?

> That model might make some sense if you think e.g. of a web application,
> [...]

Yep, that is what I had in mind, but the primary objective is really to
check whether pg is responsive or not.

> So I think a more useful model is that new queries arrive at a given
> rate, and each query is expected to finish in X milliseconds from its
> arrival time (i.e the time the query is scheduled to begin, not the time
> it was sent to the server) or it's counted as failed. If a transaction
> cannot even be started by that deadline, because the connection is still
> busy with the previous query, it's counted as failed without even
> sending it to the server. With that definition, it makes sense to
> specify the latency limit even without --rate.

Yep. But that is not what I'm doing here. It would be interesting as well.
It would be another patch.

> In that case, it's simply a limit on how long each query's
> execution is allowed to last until it's considered as failed. IOW, each
> query's scheduled start time is when the previous query ends.

Not under --rate... that is the point of throttling! Under throttling,
the latency should really be computed wrt to the schedule start time and
not the actual start time which may be 10 seconds afterwards when things
are going bad... Also, there is the question of whether the "failed query"
is executed or not. Here I'm not executing them, in effect they were
aborted by the application. With your suggestion they would be executed
anyway but considered failed.

So what you are suggesting is another (interesting) functionnality, that
could indeed be named "latency limit" (count slow above a threshold
queries), what I'm doing here is "lag limit" (scheduled query could not
start on time and are skipped, this is really specific to --rate).

In the updated patch attached, I changed the explanations, documentation
and name to "lag limit" instead of "latency limit" to clarify this point.
It was really a misnommer.

--
Fabien.