From: | Jacob Champion <jacob(dot)champion(at)enterprisedb(dot)com> |
---|---|
To: | Robert Haas <robertmhaas(at)gmail(dot)com> |
Cc: | Jelte Fennema-Nio <postgres(at)jeltef(dot)nl>, Daniel Gustafsson <daniel(at)yesql(dot)se>, Andres Freund <andres(at)anarazel(dot)de>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: RFC: adding pytest as a supported test framework |
Date: | 2024-06-18 14:34:17 |
Message-ID: | CAOYmi+=T1t6yn7seNjUR74wFwwtvysp6bgnOMwJ2AK0txU=C6A@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
(slowly catching up from the weekend email backlog)
On Fri, Jun 14, 2024 at 5:10 AM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> I mean, both Perl and Python are Turing-complete.
Tom responded to this better than I could have, but I don't think this
is a helpful statement. In fact I opened the unconference session with
it and immediately waved it away as not-the-point.
> I just don't believe in the idea that we're going to write one
> category of tests in one language and another category in another
> language.
You and I will probably not agree, then, because IMO we already do
that. SQL behavior is tested in SQL via pg_regress characterization
tests. End-to-end tests are written in Perl. Lower-level tests are
often written in C (and, unfortunately, then driven in Perl instead of
C; see above mail to Noah).
I'm fundamentally a polyglot tester by philosophy, so I don't see
careful use of multiple languages as an inherent problem to be solved.
They increase complexity (considerably so!) but generally provide
sufficient value to offset the cost.
> As soon as we open the door to Python tests, people are
> going to start writing the TAP tests that they would have written in
> Perl in Python instead.
There's a wide spectrum of opinions between yours (which I will
cheekily paraphrase as "people will love testing in Python so much
they'll be willing to reinvent all of the wheels" -- which in the
short term would increase maintenance cost but in the long term sounds
like a very good problem to have), and people who seem to think that
new test suite infrastructure would sit unused because no one wants to
write tests anyway (to pull a strawman out of some hallway
conversations at PGConf.dev). I think the truth is probably somewhere
in the middle?
My prior mail was an attempt to bridge the gap between today and the
medium term, by introducing a series of compromises and incremental
steps in response to specific fears. We can jump forward to the end
state and try to talk about it, but I don't control the end state and
I don't have a crystal ball.
> So as I see
> it, the only reasonable plan here if we want to introduce testing in
> Python (or C#, or Ruby, or Go, or JavaScript, or Lua, or LOLCODE) is
> to try to achieve a reasonable degree of parity between that language
> and Perl. Because then we can at least review the new infrastructure
> all at once, instead of incrementally spread across many patches
> written, reviewed, and committed by many different people.
I don't at all believe that a test API which is ported en masse as a
horizontal layer, without motivating vertical slices of test
functionality, is going to be fit for purpose.
And "written, reviewed, and committed by many different people" is a
feature for me, not a bug. One of the goals of the thread is to
encourage more community test writing than we currently have.
Otherwise, I could have kept silent (I am very happy with my personal
suite and have been comfortably maintaining it for a while). I am
trying to build community momentum around a pain point that is
currently rusted in place.
> Consider the meson build system project. To get that committed, Andres
> had to make it do pretty much everything MSVC could do and mostly
> everything that configure could do
I think some lessons can be pulled from that, but fundamentally that's
a port of the build infrastructure done by a person with a commit bit.
There are some pretty considerable differences. (And even then, it
wasn't "done" with the first volley of patches, right?)
Thanks,
--Jacob
From | Date | Subject | |
---|---|---|---|
Next Message | Jacob Champion | 2024-06-18 14:35:35 | Re: RFC: adding pytest as a supported test framework |
Previous Message | Andres Freund | 2024-06-18 14:32:43 | Re: Truncation of mapped catalogs (whether local or shared) leads to server crash |