Re: New regression test time

Josh,

* Josh Berkus (josh(at)agliodbs(dot)com) wrote:
> If we don't have a test for it, then we can break it in the future and
> not know we've broken it until .0 is released. Is that really a
> direction we're happy going in?

To be fair, AIUI anyway, certain companies have much larger regression
suites that they run new versions of PG against. I'm sure they don't
have the platform coverage that the buildfarm does, but I expect no
small number of build farm animals would fall over under the strain of
that regression suite (or at least, they'd have trouble keeping up with
the commit rate).

I'm definitely pro-more-tests when they add more code/feature coverage
and are nominal in terms of additional time taken during 'make check',
but I seriously doubt we're ever going to have anything close to
complete code coverage in such a suite of tests.

> I have to say, I'm really surprised at the level of resistance people on
> this list are showing to the idea of increasing test coverage. I thought
> that Postgres was all about reliability? For a project as mature as we
> are, our test coverage is abysmal, and I think I'm starting to see why.

I do think having these tests split into different groups would be
valuable. It might even be good to split them into groups based on what
code they exercise and then devs might be able to decide "I'm working in
this area right now, I want the tests that are applicable to that code".
Might be a bit too much effort though too, dunno. Would be really neat
if it was automated in some way. ;)

Thanks,

Stephen