Re: RFC: adding pytest as a supported test framework

On Wed, Jun 12, 2024 at 1:30 PM Daniel Gustafsson <daniel(at)yesql(dot)se> wrote:
> > On 12 Jun 2024, at 17:50, Andres Freund <andres(at)anarazel(dot)de> wrote:
> > On 2024-06-11 07:28:23 -0700, Jacob Champion wrote:
>
> >> The OAuth pytest suite makes extensive use of
> >> - psycopg, to easily drive libpq;
> >
> > That's probably not going to fly. It introduces painful circular dependencies
> > between building postgres (for libpq), building psycopg (requiring libpq) and
> > testing postgres (requiring psycopg).
>
> I might be missing something obvious, but if we use a third-party libpq driver
> in the testsuite doesn't that imply that a patch adding net new functionality
> to libpq also need to add it to the driver in order to write the tests? I'm
> thinking about the SCRAM situation a few years back when drivers weren't up to
> date.

Yeah, I don't think depending on psycopg2 is practical at all. We can
either shell out to psql like we do now, or we can use something like
CFFI.

On the overall topic of this thread, I personally find most of the
rationale articulated in the original message unconvincing. Many of
those improvements could be made on top of the Perl framework we have
today, and some of them have been discussed, but nobody's done the
work. I also don't understand the argument that assert a == b is some
new and wonderful thing; I mean, you can already do is($a,$b,"test
name") which *also* shows you the values when they don't match, and
includes a test name, too! I personally think that most of the
frustration associated with writing TAP tests has to do with (1)
Windows behavior being randomly different than on other platforms in
ways that are severely under-documented, (2)
PostgreSQL::Test::whatever being somewhat clunky and under-designed,
and (3) the general difficulty of producing race-free test cases. A
new test framework isn't going to solve (3), and (1) and (2) could be
fixed anyway.

However, I understand that a lot of people would prefer to code in
Python than in Perl. I am not one of them: I learned Perl in the early
nineties, and I haven't learned Python yet. Nonetheless, Python being
more popular than Perl is a reasonable reason to consider allowing its
use in PostgreSQL. But if that's the reason, let's be up front about
it.

I do think we want a scripting language here i.e. not C.

--
Robert Haas
EDB: http://www.enterprisedb.com