Re: TAP output format in pg_regress

Hi,

> Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com> writes:
> > I'm not sure what to make of all these options. I think providing a TAP
> > output for pg_regress is a good idea. But then do we still need the old
> > output? Is it worth maintaining two output formats that display exactly
> > the same thing in slightly different ways?
>
> Probably is, because this is bad:

> > ... The proposed default format now hides the
> > fact that some tests are started in parallel. I remember the last time
> > I wanted to tweak the output of the parallel tests, people were very
> > attached to the particular timing and spacing of the current output. So
> > I'm not sure people will like this.
>
> and so is this:
>
> > The timing output is very popular. Where is that in the TAP output?
>
> Both of those things are fairly critical for test development. You
> need to know what else might be running in parallel with a test case,
> and you need to know whether you just bloated the runtime unreasonably.

That should be doable with tap as well - afaics the output of that could
nearly be the same as now, preceded by a #.

The test timing output could (and I think should) also be output - but if I
read the tap specification correctly, we'd either need to make it part of the
test "description" or on a separate line.

On 2022-07-04 10:39:37 -0400, Tom Lane wrote:
> More generally, I'm unhappy about the proposal that TAP should become
> the default output. There is nothing particularly human-friendly
> about it, whereas the existing format is something we have tuned to
> our liking over literally decades. I don't mind if there's a way to
> get TAP when you're actually intending to feed it into a TAP-parsing
> tool, but I am not a TAP-parsing tool and I don't see why I should
> have to put up with it.

I'm mostly interested in the tap format because meson's testrunner can parse
it - unsurprisingly it doesn't understand the current regress output. It's a
lot nicer to immediately be pointed to the failed test(s) than having to scan
through the output "manually".

Greetings,

Andres Freund