Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>
Cc: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?
Date: 2022-08-07 07:56:00
Message-ID: Yu9wEO2X+RwoqVUW@paquier.xyz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sun, Aug 07, 2022 at 10:41:49AM +0530, Bharath Rupireddy wrote:
> Agree. I separated out the changes.

+
+/*
+ * A convenience wrapper for pwritev() that retries on partial write. If an
+ * error is returned, it is unspecified how much has been written.
+ */
+ssize_t
+pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset)

If moving this routine, this could use a more explicit description,
especially on errno, for example, that could be consumed by the caller
on failure to know what's happening.

>> +/*
>> + * A convenience wrapper for pg_pwritev_with_retry() that zero-fills the given
>> + * file of size total_sz in batches of size block_sz.
>> + */
>> +ssize_t
>> +pg_pwritev_with_retry_and_init(int fd, int total_sz, int block_sz)
>>
>> Hmm, why not give it a proper name that says it writes zeroes?
>
> Done.

FWIW, when it comes to that we have a couple of routines that just use
'0' to mean such a thing, aka palloc0(). I find 0002 confusing, as it
introduces in fe_utils.c a new wrapper
(pg_pwritev_with_retry_and_write_zeros) on what's already a wrapper
(pg_pwritev_with_retry) for pwrite().

A second thing is that pg_pwritev_with_retry_and_write_zeros() is
designed to work on WAL segments initialization and it uses
XLOG_BLCKSZ and PGAlignedXLogBlock for the job, but there is nothing
in its name that tells us so. This makes me question whether
file_utils.c is a good location for this second thing. Could a new
file be a better location? We have a xlogutils.c in the backend, and
a name similar to that in src/common/ would be one possibility.
--
Michael

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Álvaro Herrera 2022-08-07 08:21:23 Re: pgsql: BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking
Previous Message John Naylor 2022-08-07 07:47:36 Re: failing to build preproc.c on solaris with sun studio