Re: optimize file transfer in pg_upgrade

On Tue, Mar 18, 2025 at 10:12:51AM -0400, Andres Freund wrote:
> On 2025-03-18 10:04:41 -0400, Tom Lane wrote:
>> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> > I'm not quite sure what the best thing is to do is for the pg_upgrade
>> > tests in particular, and it may well be best to do as you propose for
>> > now and figure that out later. But I question whether just rerunning
>> > all of those tests with several different mode flags is the right
>> > thing to do. Why for example does 005_char_signedness.pl need to be
>> > checked under both --link and --clone? I would guess that there are
>> > one or maybe two tests in src/bin/pg_upgrade/t that needs to test
>> > --link and --clone and they should grow internal loops to do that
>> > (when supported by the local platform) and PG_UPGRADE_TEST_MODE should
>> > go in the garbage.
>>
>> +1
>>
>> I'd be particularly allergic to running 002_pg_upgrade.pl multiple
>> times, as that's one of our most expensive tests, and I flat out
>> don't believe that expending that many cycles could be justified.
>> Surely we can test these modes sufficiently in some much cheaper and
>> more targeted way.
>
> +1

Here is a first sketch at a test that cycles through all the transfer modes
and makes sure they succeed or fail with an error along the lines of "not
supported on this platform." Each test verifies that some very simple
objects make it to the new version, which we could of course expand on.
Would something like this suffice?

--
nathan