Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)

Hi,

On Sat, Nov 23, 2024 at 5:14 AM Thomas Munro <thomas(dot)munro(at)gmail(dot)com> wrote:

> On Fri, Nov 22, 2024 at 10:55 PM Jakub Wartak
> <jakub(dot)wartak(at)enterprisedb(dot)com> wrote:
> > but with attached _lseeki64 dirty patch I've got success and resulting
> tarball greater than 2^31 too:
>

>
> For ftruncate(), though you didn't see a problem with that, Jacob is
> quite right that it must be able to corrupt stuff, and I retract that
> replacement code I showed earlier, it's not really OK to move the file
> position around. I would like to use _s_chsize() instead, but it
> arrived in msvcr80.dll so I don't think old MinGW can use it. Here's
> a new idea (sketch code not tested):
>
> static inline int
> ftruncate(int fd, pgoff_t length)
> {
> #if defined(_UCRT) || defined(_MSC_VER)
> /* MinGW + UCRT and all supported MSVC versions have this. */
> errno = _s_chsize(fd, length);
> return errno == 0 ? 0 : -1;
> #else
> /* MinGW + ancient msvcrt.dll has only _chsize, limited by off_t
> (long). */
> if (length > LONG_MAX)
> {
> /* A clear error is better than silent corruption. */
> errno = EFBIG;
> return - 1;
> }
> return _chsize(fd, length);
> #endif
> }
>

With the earlier patch, there were some compilation issues on other
platforms, I have modified the patch a little by using some of the changes
from the earlier patch shared by Thomas. Also, I have used the above code
to enable ftruncate() to handle pgoff_t. All the builds and tests on the CI
build <https://cirrus-ci.com/build/6351585205288960> passed except an
already existing build failure in the mingw_cross compiler (not related to
the patch).

--
Regards,
Davinder.