On Mon, Mar 10, 2025 at 7:45 PM Melanie Plageman
<melanieplageman(at)gmail(dot)com> wrote:
>
> Attached v35 0003-0005 is the same as v34 but is on top of two new
> commits: 0001 increases the default effective_io_concurrency to 16 and
> 0002 adds a READ_STREAM_RANDOM flag which disables sequential
> detection and is used by bitmap heap scan.
Attached v36 is built on top of the read stream updates Thomas posted
earlier today in [1]. I've been doing some performance testing and
from what I see, we've closed the major gaps.
The case I was most concerned about was cyclic data (narrow bands of
sequential data in a larger random dataset) with reasonable values for
io_combine_limit, effective_io_concurrency, and OS read ahead. I've
attached a chart which compares the query time for master, my original
bitmap heap scan streaming read user "og", the bitmap heap scan
streaming read user with sequential detection turned off completely
"random", and the bitmap heap scan streaming read user with the
updated read stream changes Thomas posted today "rsupdate". You can
see that it eliminates the regression.
For some of the other cases, because we still do sequential detection
with Thomas' changes, we sometimes pause prefetching which means if
you have turned off OS read ahead, then it'll be slower than without
the read stream user. That's okay, though -- in the vastly more common
case of OS readahead being enabled, we can avoid unnecessary
prefetching and beat current master.
Overall, I feel pretty good about merging this once Thomas merges the
read stream patches.
- Melanie
[1] https://www.postgresql.org/message-id/CA%2BhUKGKH2mwj%3DmpCobC4ZyBdR4ToW2hpQcYFDivj40fxf4yQow%40mail.gmail.com
