Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)

From: Davinder Singh <davinder(dot)singh(at)enterprisedb(dot)com>
To: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Dilip Kumar <dilip(dot)kumar(at)enterprisedb(dot)com>
Subject: Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)
Date: 2024-12-07 07:42:59
Message-ID: CAEbAQ-VgeghebE+xgJFOF9dppYX0OS-7N_vq=u7fuhz5oBWyyQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Dec 6, 2024 at 11:31 AM Thomas Munro <thomas(dot)munro(at)gmail(dot)com> wrote:

> Some better new:
>
> 1. _chsize_s does in fact seem to work in an msvcrt.dll build.
> Thanks to Andres for testing that for me on Windows with a standalone
> 2 line program on ucrt and msvcrt runtimes.
>
> 2. MinGW always has _chsize_s, but it does more or less what I had
> been proposing already if it can't find the function by the Windows
> thing like dlsym, so there's no point in writing another thing like that.
> The test was with a 3G file though so it was the real function.
>
>
> https://github.com/mingw-w64/mingw-w64/blob/master/mingw-w64-crt/secapi/_chsize_s.c
>
> 3. After a long time trying various ways to make it tidy and getting
> incomprehensible results I eventually realised that precompiled
> headers were causing strange results on MinGW/Meson builds (on CI),
> due to fighting over macros with this guy:
>
>
> https://github.com/mingw-w64/mingw-w64/blob/master/mingw-w64-headers/crt/unistd.h
>
> I found it easiest to completely stop it from getting in our way
> completely, with the attached. Thoughts, anyone?
>
> Davinder, Jakub, are you in a position repro the issue and make
> versions of the patch for the 13, 15 and master branches on top of
> that?
>

Yes, Thomas, the issue is reproducible.
After applying your patch and replacing off_t with pgoff_t, the problem has
been resolved. I have attached the patches for the 13, 15, and master.
These patches include only the changes related to pgoff_t, intended for use
with the v3-0001 patch--

Regards,
Davinder.

Attachment Content-Type Size
v3_0002_PG13_Bugfix_Windows_pg_basebackup.patch application/x-patch 1.6 KB
v3_0002_master_Bugfix_Windows_pg_basebackup.patch application/x-patch 1.9 KB
v3_0002_PG15_Bugfix_Windows_pg_basebackup.patch application/x-patch 1.6 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Srinath Reddy Sadipiralla 2024-12-07 08:17:21 Re: Why we need to check for local buffers in BufferIsExclusiveLocked and BufferIsDirty?
Previous Message jian he 2024-12-07 02:13:26 Re: proposal: schema variables