Re: Test to dump and restore objects left behind by regression

Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> writes:
> Hmm, I didn't mean that we'd maintain a separate schedule. I meant that
> we'd take the existing schedule, then apply some Perl magic to it that
> grep-outs the tests that we know to contribute nothing, and generate a
> new schedule file dynamically. We don't need to maintain a separate
> schedule file.

This seems like a fundamentally broken approach to me.

The entire argument for using the core regression tests as a source of
data to test dump/restore is that, more or less "for free", we can
expect to get coverage when new SQL language features are added.
That's always been a little bit questionable --- there's a temptation
to drop objects again at the end of a test script. But with this,
it becomes a complete crapshoot whether the objects you need will be
included in the dump.

I think instead of going this direction, we really need to create a
separately-purposed script that simply creates "one of everything"
without doing anything else (except maybe loading a little data).
I believe it'd be a lot easier to remember to add to that when
inventing new SQL than to remember to leave something behind from the
core regression tests. This would also be far faster to run than any
approach that involves picking a random subset of the core test
scripts.

regards, tom lane