Re: BitmapHeapScan streaming read user and prelim refactoring

From: Melanie Plageman <melanieplageman(at)gmail(dot)com>
To: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
Cc: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Andres Freund <andres(at)anarazel(dot)de>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com>
Subject: Re: BitmapHeapScan streaming read user and prelim refactoring
Date: 2024-03-27 19:37:50
Message-ID: 20240327193750.3mlcmzqondpj27xe@liskov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Mar 25, 2024 at 12:07:09PM -0400, Melanie Plageman wrote:
> On Sun, Mar 24, 2024 at 06:37:20PM -0400, Melanie Plageman wrote:
> > On Sun, Mar 24, 2024 at 5:59 PM Tomas Vondra
> > <tomas(dot)vondra(at)enterprisedb(dot)com> wrote:
> > >
> > > BTW when you say "up to 'Make table_scan_bitmap_next_block() async
> > > friendly'" do you mean including that patch, or that this is the first
> > > patch that is not one of the independently useful patches.
> >
> > I think the code is easier to understand with "Make
> > table_scan_bitmap_next_block() async friendly". Prior to that commit,
> > table_scan_bitmap_next_block() could return false even when the bitmap
> > has more blocks and expects the caller to handle this and invoke it
> > again. I think that interface is very confusing. The downside of the
> > code in that state is that the code for prefetching is still in the
> > BitmapHeapNext() code and the code for getting the current block is in
> > the heap AM-specific code. I took a stab at fixing this in v9's 0013,
> > but the outcome wasn't very attractive.
> >
> > What I will do tomorrow is reorder and group the commits such that all
> > of the commits that are useful independent of streaming read are first
> > (I think 0014 and 0015 are independently valuable but they are on top
> > of some things that are only useful to streaming read because they are
> > more recently requested changes). I think I can actually do a bit of
> > simplification in terms of how many commits there are and what is in
> > each. Just to be clear, v9 is still reviewable. I am just going to go
> > back and change what is included in each commit.
>
> So, attached v10 does not include the new version of streaming read API.
> I focused instead on the refactoring patches commit regrouping I
> mentioned here.

Attached v11 has the updated Read Stream API Thomas sent this morning
[1]. No other changes.

- Melanie

[1] https://www.postgresql.org/message-id/CA%2BhUKGJTwrS7F%3DuJPx3SeigMiQiW%2BLJaOkjGyZdCntwyMR%3DuAw%40mail.gmail.com

Attachment Content-Type Size
v11-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch text/x-diff 2.8 KB
v11-0002-BitmapHeapScan-set-can_skip_fetch-later.patch text/x-diff 2.2 KB
v11-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch text/x-diff 15.0 KB
v11-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch text/x-diff 2.2 KB
v11-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch text/x-diff 2.3 KB
v11-0006-table_scan_bitmap_next_block-returns-lossy-or-ex.patch text/x-diff 4.4 KB
v11-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch text/x-diff 2.9 KB
v11-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch text/x-diff 4.1 KB
v11-0009-Make-table_scan_bitmap_next_block-async-friendly.patch text/x-diff 22.9 KB
v11-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch text/x-diff 16.0 KB
v11-0011-table_scan_bitmap_next_block-counts-lossy-and-ex.patch text/x-diff 5.2 KB
v11-0012-Hard-code-TBMIterateResult-offsets-array-size.patch text/x-diff 5.4 KB
v11-0013-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch text/x-diff 20.7 KB
v11-0014-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch text/x-diff 31.5 KB
v11-0015-Remove-table_scan_bitmap_next_block.patch text/x-diff 11.8 KB
v11-0016-v10-Read-Stream-API.patch text/x-diff 71.2 KB
v11-0017-BitmapHeapScan-uses-read-stream-API.patch text/x-diff 26.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Ranier Vilela 2024-03-27 19:46:57 Re: Slow GRANT ROLE on PostgreSQL 16 with thousands of ROLEs
Previous Message Maciek Sakrejda 2024-03-27 19:10:02 Re: Possibility to disable `ALTER SYSTEM`