Re: On Judging the Value of Tests

On 22 November 2017 at 08:43, Jankirk.Vincent., Jamison <
k(dot)jamison(at)jp(dot)fujitsu(dot)com> wrote:

> Dear PG Experts,
>
>
>
> I am entirely very new to PG development.
>
> Currently, I’m studying the test suites, and I feel the more experienced
> PG devs here could provide some insights on testing because I could not
> find concrete answers in the Postgres documentations.
>
>
>
> 1. How do you judge when a test suite is acceptable to be added to
> Postgres OSS source code? (How do you judge the value of a test suite?)
>

Make your argument for it, and see if others agree. There's no formal
process.

Inputs into the decision making process include:

* How much coverage of previously untested functionality it adds
* How much code coverage it adds
* How long the test takes to run, especially considering the slow buildfarm
boxes and development turnaround time
* Whether the test fits into one of the existing suites we run routinely,
or requires separate steps
* How much work will be required to maintain the test

> 3. Is there a standard way of writing tests on the source code that
> we should follow, like when should test be written in TAP/SQL/C formats and
> how long should it be?
>
> ① I know that TAP test is for client program tests, SQL for
> regression tests with sql queries, but there are instances also where tests
> are also written in C like isolation tests, etc. How do we best judge which
> language is preferred to use when writing tests for Postgres components?
> How long should a test be when proposing them to hackers page?
>

In general, prefer pg_regress if you can use it. Isolation tests for
concurrency issues. TAP tests if you can't write it with pg_regress or
isolation tester. Test modules only if you really must.

>
>
> 4. In the src/test/examples directory (which are all libpq tests),
> why is the “examples” directory not included when building postgres? (Why
> weren't these libpq tests added to src/interface/libpq/test or in
> regression test suite instead?) In short, how to know where (in which
> file/directory in source code) to put a test?
>
Dunno, sorry.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services