Contributing test cases to improve coverage

From: J F <jonathanfoo0523(at)gmail(dot)com>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Contributing test cases to improve coverage
Date: 2024-06-12 16:44:29
Message-ID: CAEFLKGhVEP4_N4T4KQ75av7CEJreJpY1j2T+Dxq96dyJ1khBhg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello All,

I am working on a project that aims to produce test cases that
improve mutation coverage of a dbms's test suite.

The rough workflow of the project goes as follows:
(a) apply mutation at a souce code level
(b) compile and check if the mutated installation passed existing testsuite
(c) If not, fuzz the mutated installation with SQL fuzzer
(d) if a fuzzor successfully produce a test case that crash or trigger bugs
in the mutated installation, use a reduction tools to reduce the test case
(e) add the reduced test case to existing test suite

For postgres, I am looking at adding test cases to test suite in
test/src/regress/. I have gone through (a)-(e), and managed to produced
some test cases. As an example, I claim the test case
```
CREATE RECURSIVE VIEW a(b) AS SELECT'' ;
SELECT FROM a WHERE NULL;
```
could kill the following mutation at optimizer/plan/setrefs.c, 502:5--502:33
Original binary operator expression:
```
rte->rtekind == RTE_SUBQUERY
````
Replacement expression:
```
(rte->rtekind) >= RTE_SUBQUERY
```

I have a few questions about adding these test cases:

(a) The regression test suite is run by a parallel scheduler, with some
test cases dependent on previous test cases. If I just add my test case as
part of the parallel scheduler’s tests, it might not work, since previous
test cases in the scheduler might already create the same table, for
instance.

(b) How do I get my test cases reviewed and ultimately included in a future
release of PostgreSQL?

Thank you for your time.

Regards,
Jon

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Daniel Gustafsson 2024-06-12 16:45:52 Re: RFC: adding pytest as a supported test framework
Previous Message Amonson, Paul D 2024-06-12 16:43:41 RE: Proposal for Updating CRC32C with AVX-512 Algorithm.