Re: BitmapHeapScan streaming read user and prelim refactoring

On Wed, Feb 28, 2024 at 8:22 AM Tomas Vondra
<tomas(dot)vondra(at)enterprisedb(dot)com> wrote:
>
> Hi,
>
> I haven't looked at the code very closely yet, but I decided to do some
> basic benchmarks to see if/how this refactoring affects behavior.
>
> Attached is a simple .sh script that
>
> 1) creates a table with one of a couple basic data distributions
> (uniform, linear, ...), with an index on top
>
> 2) runs a simple query with a where condition matching a known fraction
> of the table (0 - 100%), and measures duration
>
> 3) the query is forced to use bitmapscan by disabling other options
>
> 4) there's a couple parameters the script varies (work_mem, parallel
> workers, ...), the script drops caches etc.
>
> 5) I only have results for table with 1M rows, which is ~320MB, so not
> huge. I'm running this for larger data set, but that will take time.
>
>
> I did this on my two "usual" machines - i5 and xeon. Both have flash
> storage, although i5 is SATA and xeon has NVMe. I won't share the raw
> results, because the CSV is like 5MB - ping me off-list if you need the
> file, ofc.

I haven't looked at your results in detail yet. I plan to dig into
this more later today. But, I was wondering if it was easy for you to
run the shorter tests on just the commits before the last
https://github.com/melanieplageman/postgres/tree/bhs_pgsr
i.e. patches 0001-0013. Patch 0014 implements the streaming read user
and removes all of the existing prefetch code. I would be interested
to know if the behavior with just the preliminary refactoring differs
at all.

- Melanie