Re: BitmapHeapScan streaming read user and prelim refactoring

On Mon, Apr 7, 2025 at 7:34 PM Thomas Munro <thomas(dot)munro(at)gmail(dot)com> wrote:
>
> On Thu, Feb 13, 2025 at 1:40 PM Melanie Plageman
> <melanieplageman(at)gmail(dot)com> wrote:
> > Thomas mentioned this to me off-list, and I think he's right. We
> > likely need to rethink the way parallel bitmap heap scan workers get
> > block assignments for reading and prefetching to make it more similar
> > to parallel sequential scan. The workers should probably get
> > assignments of a range of blocks. On master, each worker does end up
> > issuing reads/fadvises for a bunch of blocks in a row -- even though
> > that isn't the intent of the parallel bitmap table scan
> > implementation. We are losing some of that with the patch -- but only
> > because it is behaving as you would expect given the implementation
> > and design. I don't consider that a blocker, though.
>
> I had a crack at this one a few weeks back, and wanted to share some
> thoughts.

Fwiw, I've had reasonably good results, in a similar situation, by
just batching + double-buffering.

Since you're using AIO (as Melanie points out) there should be no real
penalty to having a single worker request the next batch of blocks,
when the current block is used up.

Then you can separate "reading" from "prefetching": whoever reads the
last block in the current batch, prefetches the next batch.

This way, you could preserve your existing "reading" logic, and you
wouldn't need to create several new, related queues.

James