Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Nathan Bossart <nathandbossart(at)gmail(dot)com>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?
Date: 2023-02-20 05:33:22
Message-ID: Y/MGIhD8KFhfEBzz@paquier.xyz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Feb 17, 2023 at 09:31:14AM -0800, Andres Freund wrote:
> On 2023-02-17 16:19:46 +0900, Michael Paquier wrote:
>> But it looks like I misunderstood what this quote meant compared to
>> what v3 does. It is true that v3 sets iov_len and iov_base more than
>> needed when writing sizes larger than BLCKSZ.
>
> I don't think it does for writes larger than BLCKSZ, it just does more for
> writes larger than PG_IKOV_MAX * BLCKSZ. But in those cases CPU time is going
> to be spent elsewhere.

Yep.

>> Seems like you think that it is not really going to matter much to track
>> which iovecs have been already initialized during the first loop on
>> pg_pwritev_with_retry() to keep the code shorter?
>
> Yes. I'd bet that, in the unlikely case you're going to see any difference at
> all, unconditionally initializing is going to win.
>
> Right now we memset() 8KB, and iterate over 32 IOVs, unconditionally, on every
> call. Even if we could do some further optimizations of what I did in the
> patch, you can initialize needed IOVs repeatedly a *lot* of times, before it
> shows up...
>
> I'm inclined to go with my version, with the argument order swapped to
> Bharath's order.

Okay. That's fine by me.
--
Michael

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2023-02-20 05:37:42 Re: pg_upgrade and logical replication
Previous Message Kyotaro Horiguchi 2023-02-20 05:21:39 Re: Add WAL read stats to pg_stat_wal