Re: Proposal: remove obsolete hot-standby testing infrastructure

Alexander Lakhin <exclusion(at)gmail(dot)com> writes:
> 04.01.2022 00:50, Tom Lane wrote:
>> The attached proposed patch removes some ancient infrastructure for
>> manually testing hot standby. I doubt anyone has used this in years,
>> because AFAICS there is nothing here that's not done better by the
>> src/test/recovery TAP tests. (Or if there is, we ought to migrate
>> it into the TAP tests.)

> It's hardly that important, but we (Postgres Pro) run this test
> regularly to check for primary-standby compatibility. It's useful when
> checking binary packages from different minor versions. For example, we
> setup postgresql-14.0 and postgresql-14.1 aside (renaming one
> installation' directory and changing it's port) and perform the test.
> What've found with it was e.g. incompatibility due to linkage of
> different libicu versions (that was PgPro-only issue). I don't remember
> whether we found something related to PostgreSQL itself, but we
> definitely use this test and I'm not sure how to replace it in our setup
> with a TAP test. On the other hand, testing binaries is not accustomed
> in the community yet, so when such testing will be adopted, probably a
> brand new set of tests should emerge.

Oh, interesting. I definitely concur that testing compatibility of
different builds or minor versions is an important use-case. And
I concede that making src/test/recovery do it would be tricky and
a bit out-of-scope. But having said that, the hs_standby_* scripts
seem like a poor fit for the job too. AFAICS they don't really
test any user data type except integer (so I'm surprised that they
located an ICU incompatibility for you); and they spend a lot of
effort on stuff that I doubt is relevant because it *is* covered
by the TAP tests.

If I were trying to test that topic using available spare parts,
what I'd do is run the regular regression tests on the primary
and see if the standby could track it. Maybe pg_dump from both
servers afterwards and see if the results match, a la the pg_upgrade
test. Bonus points for a script that could run some other pg_regress
suite such as one of the contrib modules, because then you could
check compatibility of those too.

I'm happy to keep the hs_standby_* scripts if there's a live use-case
for them; but I don't see what they're doing for you that wouldn't be
done better by other pg_regress suites.

regards, tom lane