Vectored IO in XLogWrite()

From: Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Subject: Vectored IO in XLogWrite()
Date: 2024-08-06 09:35:54
Message-ID: CAGPVpCQr1yk6p1ZJguacosuqP+OLjXR2EX9yH3tmhM8VQK=smA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers,

I was looking into XLogWrite() and saw the below comment. It cannot really
circle back in wal buffers without needing to call pg_write() since next
pages wouldn't be contiguous in memory. So it needs to write whenever it
reaches the end of wal buffers.

/*
> * Dump the set if this will be the last loop iteration, or if we are
> * at the last page of the cache area (since the next page won't be
> * contiguous in memory), or if we are at the end of the logfile
> * segment.
> */

I think that we don't have the "contiguous pages" constraint when writing
anymore as we can do vectored IO. It seems unnecessary to write just
because XLogWrite() is at the end of wal buffers.
Attached patch uses pg_pwritev() instead of pg_pwrite() and tries to write
pages in one call even if they're not contiguous in memory, until it
reaches the page at startidx.

After quickly experimenting the patch and comparing the number of write
calls, the patch's affect can be more visible when wal_buffers is quite low
as it's more likely to circle back to the beginning. When wal_buffers is
set to a decent amount, the patch only saves a few write calls. But I
wouldn't expect any regression introduced by the patch (I may be wrong
here), so I thought it may be worth to consider.

I appreciate any feedback on the proposed change. I'd also be glad to
benchmark the patch if you want to see how it performs in some specific
cases since I've been struggling with coming up a good test case.

Regards,
--
Melih Mutlu
Microsoft

Attachment Content-Type Size
v1-0001-Use-pg_pwritev-in-XlogWrite.patch application/octet-stream 3.5 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2024-08-06 10:23:03 Remove hardcoded hash opclass function signature exceptions
Previous Message Amul Sul 2024-08-06 09:34:12 Re: Support specify tablespace for each merged/split partition