Re: Improving test coverage of extensions with pg_dump

On Thu, Sep 3, 2015 at 10:35 AM, Michael Paquier
<michael(dot)paquier(at)gmail(dot)com> wrote:
> On Thu, Sep 3, 2015 at 12:38 AM, Andres Freund wrote:
>> Isn't a full test with a separate initdb, create extension etc. a really
>> heavyhanded way to test this? I mean that's a test where the setup takes
>> up to 10s, whereas the actual runtime is in the millisecond range?
>>
>> Adding tests in this manner doesn't seem scalable to me.
>
> How to include such kind of tests is in the heart of the discussion
> since this patch has showed up. I think we are now close to 5
> different opinions with 4 different hackers on the matter, the Riggs'
> theorem applies nicely.
>
>> I think we should rather add *one* test that does dump/restore over the
>> normal regression test database. Something similar to the pg_upgrade
>> tests. And then work at adding more content to the regression test
>> database - potentially sourcing from src/test/modules.
>
> If you are worrying about run time, pg_upgrade already exactly does
> that. What would be the point to double the amount of time to do the
> same thing in two different places?

So, to summarize the state of this patch whose status is now "Waiting
on Author", we have the following possibilities:
1) Keep the test as-is, as an independent test of src/test/modules.
2) Integrate it in the test suite of src/test/regress and let
pg_upgrade make the work with dump/restore.
3) Create a new facility as src/test/modules/extensions that does a
run of the regression test suite with a set of extensions, and then a
dump/restore. For now the only extension added in the installation is
tables_fk. Future ones would be added here as well, though it is not
clear to me what they are and if we'd have some (there may be... Or
not).
4) Include it as a test of pg_dump in src/bin/pg_dump/t, with the
extension located in for example t/extensions and extend prove_check
to install as well any extensions in t/extensions.

I would still go for 1), the infrastructures included by the other
proposals may become useful depending on the types of tests that are
added in the future, but it is still unclear what those tests are, and
they may even need something else than what listed here (see that as
an example http://www.postgresql.org/message-id/54F62C3F.8070702@gmx.net)
to run properly.
Regards,
--
Michael