Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)

On Wed, Dec 4, 2024 at 7:04 AM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Fri, Nov 22, 2024 at 6:44 PM Thomas Munro <thomas(dot)munro(at)gmail(dot)com> wrote:
> > I'm not sure if we'd ever know if we broke MinGW + msvcrt.dll in the
> > back branches, ie if any computer actually existing on earth would
> > ever compile and run the second branch, and if so, be used for serious
> > database work and then be able to reach the failure case. I think an
> > honest but disappointing errno is probably OK in that hypothetical
> > case. It's hard to know how far to go when programming ghost
> > computers. Thoughts, anyone?
>
> Hypothetically speaking, suppose we just used _s_chsize everywhere
> i.e. we deleted the #if/#else/#endif in your example and everything
> between #else and #endif. I know very little about Windows, but I
> suppose what would happen is that if you tried to compile a
> thus-modified version of PostgreSQL on a very old MinGW, it would fail
> to compile, and if you compiled it on a newer machine and tried to run
> it on an older one, you'd get the equivalent of a dynamic linker
> failure when you tried to start the server. At least that's what would
> happen on Linux or macOS. Would Windows postpone the failure until we
> actually tried to jump to the nonexistent entrypoint?

I think it would fail to link, while building the server?

> In any case, one idea would be to adopt this approach and, if we heard
> of actual failures, we could then decide that the fallback path is
> needed, and if not, then we're done. Perhaps that's too relaxed an
> attitude; after all, breaking the entire server in a minor release is
> not great.

As a data point, v17 already made a similar assumption and nobody
squawked. But that was a major release. I honestly don't know who is
really using that toolchain outside CI-type systems used by people who
want more Unix/Windows toolchain harmony. It's a shame that it has
caused so many projects so much Windows/Windows disharmony over the
years, but it seems things have dramatically improved on that front.

AFAIK we don't really have any defined support criteria beyond "keep
the BF and CI green" for new development on master. Maybe you're
right and we don't need to be so cautious about the back-branches,
though it would irk me to leave the tree full of contradictions in
terms of code that is guarded by macros. In this case it's a pretty
small amount of extra code so it doesn't bother me much to maintain
it, as long as it's gone in master.

I'll have some feedback on the patch in the next few days.

> However, if you think that systems without msvcr80.dll are
> extinct in the wild, maybe it's OK. I don't know the history of these
> things, but a quick Google search suggested that maybe msvcr80.dll was
> part of the "Microsoft Visual C++ 2005 Redistributable Package". I
> don't know if that means all Windows shipped after 2005 would have it,
> but that's a long time ago.

I think it was more complicated in that time period, but is simple
again as of the past decade. Something like: all systems to this day
have had a msrvcrt.dll from the 90s as part of the OS, and all systems
have ucrt since ~2015 (Windows 10) as part of the OS, but in the
period in between, each MSVC version had its own library that was
*not* shipped with the OS and you had to ship it with your
application. Visual Studio users could do that, but the MinGW project
didn't like the terms and conditions and stuck to the old thing, and
also couldn't use the new ucrt thing for some reason, at then some
point the project forked, producing "MinGW-w64" (which we usually just
refer to as MinGW around here), and the successor focused on making
ucrt work for them, which took a while. That's what I gathered from
skim reading, anyway. A lot of stuff happened in the decades between
those two runtimes... UTF-8, hello, I mean, that was kind of a big
deal to not have!

> > (Obviously an exorcism is overdue in master, working on it...)
>
> How does what need to be done compare to the patches from Jakub and Davinder?

Already done last week, but that's only for master:

commit 1758d42446161f5dfae9b14791c5640239b86faa
Author: Thomas Munro <tmunro(at)postgresql(dot)org>
Date: Wed Nov 27 22:34:11 2024 +1300

Require ucrt if using MinGW.