Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Dilip Kumar <dilip(dot)kumar(at)enterprisedb(dot)com>
Subject: Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)
Date: 2025-01-06 16:22:53
Message-ID: 69ae171b-87b5-463e-8f60-671a083d2e63@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


On 2024-12-07 Sa 2:42 AM, Davinder Singh wrote:
>
> On Fri, Dec 6, 2024 at 11:31 AM Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
> wrote:
>
> Some better new:
>
> 1.  _chsize_s does in fact seem to work in an msvcrt.dll build.
> Thanks to Andres for testing that for me on Windows with a standalone
> 2 line program on ucrt and msvcrt runtimes.
>
> 2.  MinGW always has _chsize_s, but it does more or less what I had
> been proposing already if it can't find the function by the Windows
> thing like dlsym, so there's no point in writing another thing
> like that.
> The test was with a 3G file though so it was the real function.
>
> https://github.com/mingw-w64/mingw-w64/blob/master/mingw-w64-crt/secapi/_chsize_s.c
>
> 3.  After a long time trying various ways to make it tidy and getting
> incomprehensible results I eventually realised that precompiled
> headers were causing strange results on MinGW/Meson builds (on CI),
> due to fighting over macros with this guy:
>
> https://github.com/mingw-w64/mingw-w64/blob/master/mingw-w64-headers/crt/unistd.h
>
> I found it easiest to completely stop it from getting in our way
> completely, with the attached.  Thoughts, anyone?
>
> Davinder, Jakub, are you in a position repro the issue and make
> versions of the patch for the 13, 15 and master branches on top of
> that?
>
>
> Yes, Thomas, the issue is reproducible.
> After applying your patch and replacing off_t with pgoff_t, the
> problem has been resolved. I have attached the patches for the 13, 15,
> and master. These patches include only the changes related to pgoff_t,
> intended for use with the v3-0001 patch--
>
>

Hi, Thomas,

Do you have a plan for moving ahead with this?

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2025-01-06 16:28:39 Re: AIO v2.0
Previous Message Andrey M. Borodin 2025-01-06 15:57:40 Re: Sample rate added to pg_stat_statements