Re: RFC: adding pytest as a supported test framework

On 2024-06-12 We 18:34, Melanie Plageman wrote:
> On Tue, Jun 11, 2024 at 8:05 AM Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
>>
>> On 2024-06-10 Mo 21:49, Andres Freund wrote:
>>
>> Hi,
>>
>> On 2024-06-10 16:46:56 -0400, Andrew Dunstan wrote:
>>
>> I'm not sure what part of the testing infrastructure you think is
>> unmaintained. For example, the last release of Test::Simple was all the way
>> back on April 25.
>>
>> IPC::Run is quite buggy and basically just maintained by Noah these days.
>>
>>
>> Yes, that's true. I think the biggest pain point is possibly the recovery tests.
>>
>> Some time ago I did some work on wrapping libpq using the perl FFI module. It worked pretty well, and would mean we could probably avoid many uses of IPC::Run, and would probably be substantially more efficient (no fork required). It wouldn't avoid all uses of IPC::Run, though.
>>
>> But my point was mainly that while a new framework might have value, I don't think we need to run out and immediately rewrite several hundred TAP tests. Let's pick the major pain points and address those.
> FWIW, I felt a lot of pain trying to write recovery TAP tests with
> IPC::Run's pumping functionality. It was especially painful (as
> someone who knows even less Perl than the "street fighting Perl"
> Thomas Munro has described having) before the additional test
> infrastructure was added in BackgroudPsql.pm last year. As an example
> of the "worst case", it took me two full work days to go from a repro
> (with psql sessions on a primary and replica node) of the vacuum hang
> issue being explored in [1] to a sort-of working TAP test which
> demonstrated it - and that was with help from several other
> committers. Granted, this is a complex case.

The pump stuff is probably the least comprehensible and most fragile
part of the whole infrastructure. As I just mentioned to Andres, I'm
hoping to make a substantial improvement in that area.

>
> A small part of the issue is that, as Tristan has said elsewhere,
> there aren't good developer tool integrations that I know about for
> Perl. I use neovim's LSP support for C and Python (in other projects),
> and there is a whole ecosystem of tools I can use for both C and
> Python. I know not everyone likes or needs these, but I find that they
> help me write and debug code faster.

You might find this useful:
<https://climatechangechat.com/setting_up_lsp_nvim-lspconfig_and_perl_in_neovim.html>

(I don't use neovim - I'm an old emacs dinosaur.)

>
> I had offered to take a stab at writing some of the BackgroundPsql
> test infrastructure in Python. I haven't started exploring that yet or
> looking at what Jacob has done so far, but I am optimistic that this
> is an area where it is worth seeing what is available to us outside of
> IPC::Run.

Yeah, like I said, I'm working on reducing our reliance on especially
the more fragile parts of IPC::Run.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com