Re: 8.5 release timetable, again

On Thu, Aug 27, 2009 at 11:29 AM, Tom Lane<tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> Well, I wasn't suggesting adding a lot more testing of things that
>> we're already testing.  I was assuming that we would craft the
>> additional tests to hit areas that we are not now covering well.  My
>> point here is only to what Peter said upthread: we want to be able to
>> get positive results rather than waiting for "enough" negative results
>> (whatever that means).  To get positive results, you must have a test
>> suite.  While letting beta testers test whatever they want has some
>> value, testing things we think might be likely hiding places for bugs
>> (such as WAL recovery) has merit, too.  Making those tests
>> well-organized and easily repeatable is, IMHO, a Good Thing.
>
> The problem here is the "easily repeatable" bit.  Almost by definition,
> easily repeatable tests don't find hard-to-reproduce problems.  I don't
> mean to suggest that they're without value, but they are no substitute
> for beta testers doing unpredictable things.

I think you're overstating the case, but I don't want to argue the
point, particularly. What I do want to do is find a way to address
the problem described in the last sentence of this email:

http://archives.postgresql.org/pgsql-hackers/2009-08/msg01614.php

Both committers and non-committers are currently suffering from the
fact that there is not a lot of time in the year which is set aside
for development, i.e. neither CommitFest-time nor beta-time. To fix
this problem, we can:

1. Make CommitFests shorter.
2. Make CommitFests less frequent.
3. Continue developing during CommitFests.
4. Make beta cycles shorter.
5. Make beta cycles less frequent (i.e. lengthen the release cycle).
6. Continue developing during beta.

I believe (1) to be completely impractical and (3) to be
self-defeating. I suspect (2) will backfire badly. That doesn't
leave us with a lot of options. We can certainly do (5), but the
downside is that features that get committed won't hit release for a
very long time. I and others have suggested a couple of possible
approaches toward (4) or (6), such as changing the way we do release
notes, adding more regression tests to give us more (not perfect)
confidence that the release is solid, and/or branching the tree during
beta. None of those ideas have gotten a single vote of confidence
from you or Bruce. What's your suggestion?

...Robert