Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)

On Fri, Nov 22, 2024 at 8:59 AM Jacob Champion
<jacob(dot)champion(at)enterprisedb(dot)com> wrote:
> On Thu, Nov 21, 2024 at 5:53 AM Thomas Munro <thomas(dot)munro(at)gmail(dot)com> wrote:
> > The lseek() is suspicious too,
> > and might need to be redirected to _lseeki64().
>
> There's a call to ftruncate() in there too. Looks like its Windows
> definition is also 32-bit:
>
> #define ftruncate(a,b) chsize(a,b)

Ah, yes. I wrote a draft fix for those two and more in the 0002
patch[1] for another project[2].

(I'm still planning on proposing the main idea in that thread --
allowing large files for relations -- but I was talked out of doing a
generalised pgoff_t-ification drilling through loads of layers all
over our tree just for Windows, so the idea was that Windows would
simply not get the large file support, and I dropped that patch from
that project. But you can see the functions with that problem that I
found, and how I thought we should fix them. Obviously I didn't know
we still had more bugs around existing ways to make large files, which
various people have tackled over the years, so there are some
functions that are 64 bit capable like stat, just not the full set...)

As for "could not close ...", I guess dodgy data type maths must be
causing tar_close() to return -1 while trying to compute padding or
something like that, it's not literally close() that is failing.

[1] https://www.postgresql.org/message-id/attachment/146666/0002-Use-pgoff_t-in-system-call-replacements-on-Windows.patch
[2] https://www.postgresql.org/message-id/CA%2BhUKG%2BBGXwMbrvzXAjL8VMGf25y_ga_XnO741g10y0%3Dm6dDiA%40mail.gmail.com