Re: TAP backpatching policy

Robert Haas wrote:
> On Tue, May 30, 2017 at 9:12 PM, Craig Ringer <craig(at)2ndquadrant(dot)com> wrote:
> > Thoughts? Backpatch new TAP methods, etc, into back branches routinely?
>
> So, on the one hand, it is certainly useful to be able to commit tests
> to back-branches as well as to master, and it's hard to do that if the
> infrastructure isn't there.

Agreed.

> On the other hand, we tell our users that we only back-patch security
> and stability fixes.

I think being strict about what we backpatch for the production code is
a valued principle. I am not sure that not backpatching new
test-harness features is valued in the same way. One possible problem
is that the new tests cause test failures, and thus failure to create
packages for some platforms. That would be a net loss. But it won't
corrupt your data and it won't make your applications incompatible.

> It is useful to be able to show a customer a list of things that were
> done in a minor release and see nothing in there that can be described
> as optional tinkering.

In my experience, some customers are going to take our word that we've
done a good job keeping the branch clean of unnecessary changes, and
others are going to cherry-pick individual fixes regardless of what's in
there. The percentages in each group might or might not change slightly
if they see some new Perl test code, but I doubt it'd change
dramatically.

My main concern is how widely is the buildfarm going to test the new
test frameworks. If we backpatch PostgresNode-based tests to 9.2, are
buildfarm animals going to need to be reconfigured to use
--enable-tap-tests? (9.2 and 9.3 do not have --enable-tap-tests) If the
old branches need to be reconfigured, then the tests are not going to
run on a majority of animals, and that is going to cause pain on obscure
platforms that three months from now we figure didn't get any buildfarm
coverage. But if they start picking up the new tests without need for
human intervention, then the coverage should be decent enough that it
shouldn't be a problem. So my vote is that if a majority of buildfarm
animals pick up PostgresNode tests without reconfiguration, +1 for
backpatching; otherwise -1.

Being unable to backpatch tests for bugfixes is a net loss, IMO.

--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services