Re: Guidance Needed for Testing PostgreSQL Patch (CF-5044)

Hi,

On 11/29/24 12:04, postgresql_contributors wrote:
> Hi,
> I recently installed and tested the CF-5044 patch using the query
> provided on the PostgreSQL community page but could not confirm its
> functionality. I followed the standard process:
>
> 1. Applied the patch using  |git|.
> 2. Rebuilt PostgreSQL from source (|./configure|, |make|, |make install|).
> 3. Tested with the suggested query after restarting PostgreSQL.
>
> I would appreciate your guidance on verifying patch functionality and
> ensuring compliance with community testing standards. If there are
> additional steps or tools I should use, please let me know.
> Thank you for your support as I represent Newt Global in contributing to
> the PostgreSQL community.

It's not clear to me if you're asking about how to test a particular
patch (I suppose by CF 5044 you mean [1]?), or testing in general.

I don't know what you mean by "suggested query" and why you feel you
could not confirm the functionality of the patch. Can you elaborate?

FWIW the queries mentioned in the e-mail thread are usually included as
a demonstration of what the patch is meant to do, not as a definitive
proof that the patch is working. Likewise, it's not enough to just run
the tests included in the patch - we have automated tools to do that.

Testing a patch usually requires inventing new tests, thinking about
cases the author might not have considered, etc. There's no "standard"
to do that, it's often a surprisingly creative task, and it depends on
what the patch is meant to do.

Maybe start by asking questions about the patch:

- Does the patch address a real problem? Is it useful?

- Does it reliably do what it's supposed to do? Maybe there are false
positives/negatives the author did not consider?

- Could it have some unintended consequences?

- Is is properly testing the change? Maybe it's missing some tests?

- ... and so on ...

Yes, it's going to be very specific to a given patch. I don't think
there's a great "howto" for any patch :-(

Does this answer your question?

[1] https://commitfest.postgresql.org/50/5044/

--
Tomas Vondra