Re: RFC: adding pytest as a supported test framework

On Sat, Jun 15, 2024 at 12:48 PM Jelte Fennema-Nio <postgres(at)jeltef(dot)nl> wrote:
> Honestly, my primary *objective* complaint about our current test
> suite, is that when a test fails, it's very often impossible for me to
> understand why the test failed, by only looking at the output of
> "meson test". I think logging the postgres log to stderr for Perl, as
> you proposed, would significantly improve that situation. I think the
> only thing that we cannot get from Perl Test::More that we can from
> pytest, is the fancy recursive introspection of the expression that
> pytest shows on error.

This surprises me. I agree that the current state of affairs is kind
of annoying, but the contents of regress_log_whatever are usually
quite long. Printing all of that out to standard output seems like
it's just going to flood the terminal with output. I don't think I'd
be a fan of that change.

I think I basically agree with all the nearby comments about how the
advantages you cite for Python aren't, I don't know, entirely
compelling. Switching from ok() to is() or cmp_ok() or like() is minor
stuff. Where the output goes is minor stuff. The former can be fixed,
and the latter can be worked around with scripts and aliases. The one
thing I know about that *I* think is a pretty big problem about Perl
is that IPC::Run is not really maintained. But I wonder if the
solution to that is to do something ourselves instead of depending on
IPC::Run. Beyond that, I think this is just a language popularity
contest.

--
Robert Haas
EDB: http://www.enterprisedb.com