Re: 2018-03 Commitfest Summary (Andres #1)

Hello Andres,

>> For instance, I used extensively tps throttling, latencies and timeouts
>> measures when developping and testing the checkpointer sorting & throttling
>> patch.
>
> That doesn't say that much about proposed feature additions, we didn't
> say that feature isn't useful?

Sure.

The point I am trying to make is that adding capabilities to pgbench
enables new kind of performance tests, and how useful they are cannot be
foreseen from the starting point. Moreover, the usability increases when
these capabilities can be combined.

For the latency & timeout measures, my initial point was to check the
state of postgres for a performance subject I'm interested in, with the
idea that people put too much emphasis on tps and not enough on latency. I
knew there were some issues there, but I had no idea how terrible it was
before I was able to get detailed measures. So the actual process was add
capabilities (a lot of argumentation because people do not see the
point...), then use them to collect data, spot a larger than expected
problem and then try to fix it.

>> I can stop doing both if the project decides that improving pgbench
>> capabilities is against its interest.
>
> That's not what we said. There's a difference between "we do not want to
> improve pgbench" and "the cost/benefit balance seems to have shifted
> over time, and the marginal benefit of proposed features isn't that
> high".

I'm probably exagerating a bit for the sake of argument, but I still think
that the project is over conservative about pgbench feature set, which
IMHO is "nearly there", but not quite, so as to be useful for running
actual benchmarks against postgres.

I've been pushing for a consistent set of basic features. Pgbench got
boolean expressions, but they cannot be used in a conditional (eh, no \if,
low marginal benefit) and they cannot test anything related to the data
(no \gset, low marginal benefit). The current result is a half-baked
inconsistent tool, which is just too bad.

>> As pgbench patches can stay ready-to-committers for half a dozen CF, I'm not
>> sure the strain on the committer time is that heavy:-)
>
> That's just plain wrong. Even patches that linger cost time and attention.

Hmmm. A few seconds per month to move it to the next CF? A few seconds per
month so skip these few "ready" patches while reading the very long list
cluttered with a hundred "needs review" items anyway?

> [...] I'm exactly of the opposite opinion. Submitting things out of
> context, without seeing at least drafts of later patches, is a lot more
> work and doesn't allow to see the big picture.

Hmmm.

The (trivial) big picture is to allow client-side expressions in psql
(which as a \if:-) by reusing pgbench expression engine, so that one could
write things like

\let i :j + 12 * :k

or

\if :VERSION_NUM < 140000

In a psql script. For this purpose, the expression engine must somehow
support the existing syntax, including testing whether a variable exists,
hence the small 30-lines (including doc & tests) patch submission.

Obviously I should check first to get at least a committer's agreement
that the feature is desirable: I have no issue with having a patch
rejected after a long process because of bad programming and/or bad
design, but if the thing is bared from the beginning there is no point in
trying.

>> So I'm doing things one (slow) step at a time, especially as each time
>> I've submitted patches which were doing more than one thing I was asked
>> to disentangle features and restructuring.
>
> There's a difference between maintaining a set of patches in a queue,
> nicely split up, and submitting them entirely independently.

Sure. I had a bad experience before on that subject, but I may be
masochistic enough to try again:-)

--
Fabien.