Re: optimize file transfer in pg_upgrade

Hello Nathan,

28.04.2025 18:15, Nathan Bossart wrote:
> I see a couple of other pg_upgrade failures on drongo and fairywren that
> look similar, although these are for different tests:
>
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2025-03-10%2019%3A26%3A35
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2025-03-30%2013%3A03%3A05

Yeah, I've categorized the first one as [1], but now I see that it's
something different, because "pg_upgrade_output.d/ removed after
successful pg_upgrade" is not the only (and not the first) failure there.
Will fix. As to the second one, yes, it's similar in the sense that the
failed test log doesn't contain information needed to understand the
cause.

>> Moreover, even when pg_upgrade succeeds, IPC::Run::run inside
>> command_ok_or_fails_like() returns false, as we can see from a
>> successful test run:
>> https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=fairywren&dt=2025-04-27%2001%3A03%3A06&stg=misc-check
>>
>> pgsql.build/testrun/pg_upgrade/006_transfer_modes/log/regress_log_006_transfer_modes
>> [01:18:38.210](21.036s) ok 1 - pg_upgrade with transfer mode --clone: stdout matches
>> [01:18:38.211](0.001s) ok 2 - pg_upgrade with transfer mode --clone: stderr matches
> That's expected for platforms that don't support all of the modes. We
> verify the output matches a known error message in that case.

Yes, I meant that in that case we can't determine whether to preserve logs
of the failed pg_upgrade outside of command_ok_or_fails_like().

>> So maybe it's worth to adjust the test somehow to have interesting logs
>> left after a failure?
> I see some other discussion about failures with similar symptoms [0] [1].
> Commit 6f97ef0 seems to have helped with one of the tests, and there is a
> proposed patch in the latest thread [2] that AFAICT aims to fix the
> underlying issue.

Thank you for the references! Unfortunately I still can't see where the
lack of upgrade log files is discussed.

In other words, if we had logs like in the case [2], it could be helpful.

[1]
https://wiki.postgresql.org/wiki/Known_Buildfarm_Test_Failures#Upgrade_tests_fail_on_Windows_due_to_pg_upgrade_output.d.2F_not_removed
[2] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=copperhead&dt=2025-02-20%2017%3A01%3A23

Best regards,
Alexander Lakhin
Neon (https://neon.tech)