Re: TAP backpatching policy

On 1 June 2017 at 01:13, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Alvaro Herrera <alvherre(at)2ndquadrant(dot)com> writes:
>> My main concern is how widely is the buildfarm going to test the new
>> test frameworks. If we backpatch PostgresNode-based tests to 9.2, are
>> buildfarm animals going to need to be reconfigured to use
>> --enable-tap-tests?
>
> Yes. The animals that are doing it at all are using code more or less
> like this:
>
> if ($branch eq 'HEAD' or $branch ge 'REL9_4')
> {
> push(@{$conf{config_opts}},"--enable-tap-tests");
> }
>
> (verbatim from longfin's config script)
>
> So maybe that's an argument for not going back before 9.4. OTOH,
> you may be giving the buildfarm owners too little credit for
> willingness to update their configurations.

Honestly, it didn't occur to me to back-patch past the introduction of
TAP in 9.4. I was more thinking of bringing that up to current
functionality, and trying to maintain that in future so that TAP tests
in extensions, test scripts for bugs, etc could be easily used on all
back branches.

I don't have a particular objection to doing so, but initially I was
really aiming for bringing 9.5 and 9.6 up to pg10 level, since
PostgresNode is already present in 9.5 so it's a much simpler target
for backporting the pg10 stuff. Then maintaining from there going
forward, so by the time pg12 is out, everything has solid and pretty
consistent TAP infrastructure.

I'm not too fussed if everyone decides it's all too hard / not worth
it. I'll just extract src/test/modules into a separate github repo.
For use in extensions I'll teach it how to overwrite the stock
PostgresNode etc in a Pg install tree. For use for in-tree testing
it'd have a Makefile that finds and clobbers the in-tree
PostgresNode.pm etc. So it's a hassle, but not the end of the world.

I just suspect we'll all benefit from making it easier to write tests
that work across more releases, and that updating the test modules in
back branches isn't an unduly invasive thing to do.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services