Re: Create shorthand for including all extra tests

On 04.09.23 22:30, Tom Lane wrote:
> Noah Misch <noah(at)leadboat(dot)com> writes:
>> On Mon, Sep 04, 2023 at 08:16:44PM +0200, Daniel Gustafsson wrote:
>>> On 4 Sep 2023, at 17:01, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>>> I think this is a seriously bad idea. The entire point of not including
>>>> certain tests in check-world by default is that the omitted tests are
>>>> security hazards, so a developer or buildfarm owner should review each
>>>> one before deciding whether to activate it on their machine.
>
>> Other than PG_TEST_EXTRA=wal_consistency_checking, they have the same hazard:
>> they treat the loopback interface as private, so anyone with access to
>> loopback interface ports can hijack the test. I'd be fine with e.g.
>> PG_TEST_EXTRA=private-lo activating all of those. We don't gain by inviting
>> the tester to review the tests to rediscover this common factor.
>
> Yeah, I could live with something like that from the security standpoint.
> Not sure if it helps Nazir's use-case though. Maybe we could invent
> categories that can be used in place of individual test names?
> For now,
>
> PG_TEST_EXTRA="needs-private-lo slow"
>
> would cover the territory of "all", and I think it'd be very seldom
> that we'd have to invent new categories here (though maybe I lack
> imagination today).

At least the kerberos tests also appear to require a lot of randomness
for their setup, and sometimes in VM environments they hang for minutes
until they get that. I suppose that would go under "slow".

Also, at least in my mind, when we added the kerberos and ldap tests, a
partial reason for excluding them from the default run was "requires
additional unusual software to be installed". The additional kerberos
and ldap server software used in those tests is not covered by
configure/meson, so it's a bit more DIY.