Re: Streaming I/O, vectored I/O (WIP)

From: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
To: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Melanie Plageman <melanieplageman(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>
Subject: Re: Streaming I/O, vectored I/O (WIP)
Date: 2024-03-12 13:02:35
Message-ID: CA+hUKGLJi+c5jB3j6UvkgMYHky-qu+LPCsiNahUGSa5Z4DvyVA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Mar 12, 2024 at 11:39 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> + actual_nblocks = operation->nblocks = *nblocks = i + 1;
> (Dilip: I think we should break after this?)

In the next loop, i < actual_nblocks is false so the loop terminates.
But yeah that was a bit obscure, so I have added an explicit break.

Here also is a new version also of the streaming_read.c patch. This
is based on feedback from the bitmap heap scan thread, where Tomas and
Melanie noticed some problems when comparing effective_io_concurrency
= 0 and 1 against master. The attached random.sql exercises a bunch
of random scans with different settings, and random.txt shows the
resulting system calls, unpatched vs patched. Looking at it that way,
I was able to tweak the coding until I had the same behaviour as
master, except with fewer system calls wherever possible due to
coalescing or suppressing sequential advice.

Attachment Content-Type Size
random.txt text/plain 275.1 KB
v7-0001-Provide-vectored-variant-of-ReadBuffer.patch application/octet-stream 33.9 KB
v7-0002-Provide-API-for-streaming-reads-of-relations.patch application/octet-stream 24.8 KB
v7-0003-Use-streaming-reads-in-pg_prewarm.patch application/octet-stream 2.6 KB
v7-0004-WIP-Use-streaming-reads-in-heapam-scans.patch application/octet-stream 13.1 KB
v7-0005-WIP-Use-streaming-reads-in-nbtree-vacuum-scan.patch application/octet-stream 5.9 KB
v7-0006-WIP-Use-streaming-reads-in-bitmap-heapscan.patch application/octet-stream 51.1 KB
v7-0007-WIP-pg_buffercache_invalidate.patch application/octet-stream 6.4 KB
random.sql application/octet-stream 2.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Aleksander Alekseev 2024-03-12 13:03:32 Re: [PATCHES] Post-special page storage TDE support
Previous Message Peter Eisentraut 2024-03-12 12:47:43 Re: Make attstattarget nullable