Re: 8.5 release timetable, again

On Thu, Aug 27, 2009 at 11:29:42AM -0400, Tom Lane wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> > Well, I wasn't suggesting adding a lot more testing of things that
> > we're already testing. I was assuming that we would craft the
> > additional tests to hit areas that we are not now covering well. My
> > point here is only to what Peter said upthread: we want to be able to
> > get positive results rather than waiting for "enough" negative results
> > (whatever that means). To get positive results, you must have a test
> > suite. While letting beta testers test whatever they want has some
> > value, testing things we think might be likely hiding places for bugs
> > (such as WAL recovery) has merit, too. Making those tests
> > well-organized and easily repeatable is, IMHO, a Good Thing.
>
> The problem here is the "easily repeatable" bit. Almost by definition,
> easily repeatable tests don't find hard-to-reproduce problems. I don't
> mean to suggest that they're without value, but they are no substitute
> for beta testers doing unpredictable things.

I've wondered before about using a system emulator to snapshot the disk
on each write (I'd expect you could put some pretty low level hooks
in with qemu and gdb) and then run each snapshot in another system to
make sure that either the transaction is rolled back or committed as
appropriate. I guess this would take a while to run but may help catch
some obscure bugs. Or this an area that's well tested already?

--
Sam http://samason.me.uk/