Re: Postgres alpha testing docs and general test packs

On Wednesday 28 October 2009 3:55:02 pm Thom Brown wrote:
> 2009/10/28 Adrian Klaver <aklaver(at)comcast(dot)net>:
> > ----- "Guillaume Lelarge" <guillaume(at)lelarge(dot)info> wrote:
> >> Le mercredi 28 octobre 2009 à 15:13:06, Thom Brown a écrit :
> >> > Similarly: "Fix encoding handling in binary input function of xml
> >> > type." What was the problem before?
> >
> > See attached screen shot for one possible solution.
>
> In other words we need to scour the committers mailing list to hunt
> for this information? This is exactly my point. Testing doesn't
> appear to be well organised. In my last place of work we had a set of
> requirements, technical solution design and a test guide which
> instructed testers on what areas need testing. From these a test plan
> was built to ensure that the requirements were met, and that the
> technical solution was working as specified. In addition to this they
> performed regression testing in the affected areas to ensure
> everything else still worked as expected and wasn't negatively
> affected by the new changes.

On the database side this handled by 'make check' which runs a regression suite
against the source. So this would be the first thing to do to ensure that the
database is not affected by a regression when compiled in your particular
environment. As to your particular needs/application things are a bit more
involved as you mention below. Here to date it seems most people have scanned
the change list for items that affected them and then dug deeper to get the
particulars. One of the benefits/problems of Open Source ,some assembly
required.

>
> All we have are a summary of changes. We can find out all the
> information if we do plenty of searching of mailing lists and
> comparing old and new documentation, but obviously this can be
> off-putting and is duplicated for everyone who wants to participate in
> testing.
>
> I'm suggesting that while this is technically sufficient, it might be
> a better idea to provide a clear technical document of the changes
> that have been committed.

This can be seen as an opportunity to participate in the project. I am sure
plenty of people would be grateful if you where to spearhead just such a
document :)

>
> Such documentation may also potentially be reused when the final
> version is released for end-users to review for any changes they might
> need to make to their existing code and queries to ensure they don't
> break.
>
> Obviously PostgreSQL has survived very well without this, but I would
> expect this would help more users perform more testing.
>
> Thom

As was mentioned in another post the whole Alpha release program is new, so it
is still in the learning curve stage. My experience with the Postgres project
is that most itches do get scratched. It just does not always happen as fast as
everybody would like.

--
Adrian Klaver
aklaver(at)comcast(dot)net