Re: Two weeks to feature freeze

> -----Original Message-----
> From: Christopher Kings-Lynne [mailto:chriskl(at)familyhealth(dot)com(dot)au]
> Sent: Friday, June 20, 2003 10:14 PM
> To: Dann Corbit
> Cc: Jason Earl; PostgreSQL-development
> Subject: Re: [HACKERS] Two weeks to feature freeze
>
>
> > We (at CONNX Solutions Inc.) have a formal release procedure that
> > includes many tens of thousands of automated tests using dozens of
> > different platforms. There are literally dozens of
> machines (I would
> > guess 70 or so total) running around the clock for 7 days before we
> > even know if we have a release candidate. The QA team is distinct
> > from the development team, and if they say "FLOP!" the release goes
> > nowhere. No formal release until QA passes it.
>
> PostgreSQL has a comprehensive regression suite that is run
> by the developers all the time...

If you mean the one that comes with PostgreSQL, then I think the MySQL
test is better. The PostgreSQL test seems to focus more on extensions
than anything else.

> > If there is no procedure for PostgreSQL of this nature, then there
> > really needs to be. I am sure that MySQL must have
> something in place
> > like that. Their "Crash-Me" test suite shows (at least) that they
> > have put a large effort into testing.
>
> No, it means they've put a crap effort into trying to make
> other databases look bad...

It does not achieve that goal.

Most of the criticism leveled at their efforts sound like fearful hand
waving to me. True, I have not studied the test as carefully as others
have. But the PostgreSQL test is not superior to the MySQL test. I
have put considerable effort into the PostgreSQL regression test. We
achieved 100% success on the Win32 platform, including dynamic loading
of functions.