Allow io_combine_limit up to 1MB

From: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Cc: Tomas Vondra <tomas(at)vondra(dot)me>
Subject: Allow io_combine_limit up to 1MB
Date: 2025-02-11 00:12:17
Message-ID: CA+hUKG+2T9p-+zM6Eeou-RAJjTML6eit1qn26f9twznX59qtCA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Tomas queried[1] the limit of 256kB (or really 32 blocks) for
io_combine_limit. Yeah, I think we should increase it and allow
experimentation with larger numbers. Note that real hardware and
protocols have segment and size limits that can force the kernel to
split your I/Os, so it's not at all a given that it'll help much or at
all to use very large sizes, but YMMV. I was originally cautious
because I didn't want to make a few stack buffers too big, but arrays
of BlockNumber, struct iovec, and pointer don't seem too excessive at
say 128 (cf whole blocks on the stack, a thing we do, which would
still be many times larger that the relevant arrays). I was also
anticipating future code that would need to multiply that number by
other terms to allocate shared memory, but after some off-list
discussion, that seems OK: such code should be able to deal with that
using GUCs instead of maximally pessimal allocation. 128 gives a nice
round number of 1M as a maximum transfer size, and comparable systems
seem to have upper limits around that mark. Patch attached.

[1] https://www.postgresql.org/message-id/flat/85696b8e-f1bf-459e-ba97-5608c644c185%40vondra.me#a4c98be8b55095ce14897dab4793c255

Attachment Content-Type Size
0001-Increase-io_combine_limit-maximum-to-1MB.patch text/x-patch 5.7 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Sami Imseih 2025-02-11 00:14:14 Re: Proposal to CREATE FOREIGN TABLE LIKE
Previous Message Jonathan S. Katz 2025-02-11 00:09:57 2025-02-13 release announcement draft