Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)

From: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Davinder Singh <davinder(dot)singh(at)enterprisedb(dot)com>, Dilip Kumar <dilip(dot)kumar(at)enterprisedb(dot)com>
Subject: Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)
Date: 2024-12-06 06:01:06
Message-ID: CA+hUKGK0+5PAfW9q_4z9XEiVfjgcCi2JgcNBmiRxTPCpxdFMdQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Some better new:

1. _chsize_s does in fact seem to work in an msvcrt.dll build.
Thanks to Andres for testing that for me on Windows with a standalone
2 line program on ucrt and msvcrt runtimes.

2. MinGW always has _chsize_s, but it does more or less what I had
been proposing already if it can't find the function by the Windows
thing like dlsym, so there's no point in writing another thing like that.
The test was with a 3G file though so it was the real function.

https://github.com/mingw-w64/mingw-w64/blob/master/mingw-w64-crt/secapi/_chsize_s.c

3. After a long time trying various ways to make it tidy and getting
incomprehensible results I eventually realised that precompiled
headers were causing strange results on MinGW/Meson builds (on CI),
due to fighting over macros with this guy:

https://github.com/mingw-w64/mingw-w64/blob/master/mingw-w64-headers/crt/unistd.h

I found it easiest to completely stop it from getting in our way
completely, with the attached. Thoughts, anyone?

Davinder, Jakub, are you in a position repro the issue and make
versions of the patch for the 13, 15 and master branches on top of
that?

Attachment Content-Type Size
v3-0001-Redirect-to-64-bit-ftruncate-and-lseek-on-Windows.patch application/x-patch 2.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Smith 2024-12-06 06:16:33 Re: Parallel heap vacuum
Previous Message Devanga.Susmitha@fujitsu.com 2024-12-06 05:54:15 Popcount optimization using SVE for ARM