Re: BitmapHeapScan streaming read user and prelim refactoring

From: Melanie Plageman <melanieplageman(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com>
Subject: Re: BitmapHeapScan streaming read user and prelim refactoring
Date: 2024-02-27 14:22:30
Message-ID: 20240227142230.nu3ytvcjwouvczlt@liskov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Feb 26, 2024 at 08:50:28PM -0500, Melanie Plageman wrote:
> On Fri, Feb 16, 2024 at 12:35:59PM -0500, Melanie Plageman wrote:
> > In the attached v3, I've reordered the commits, updated some errant
> > comments, and improved the commit messages.
> >
> > I've also made some updates to the TIDBitmap API that seem like a
> > clarity improvement to the API in general. These also reduce the diff
> > for GIN when separating the TBMIterateResult from the
> > TBM[Shared]Iterator. And these TIDBitmap API changes are now all in
> > their own commits (previously those were in the same commit as adding
> > the BitmapHeapScan streaming read user).
> >
> > The three outstanding issues I see in the patch set are:
> > 1) the lossy and exact page counters issue described in my previous
> > email
>
> I've resolved this. I added a new patch to the set which starts counting
> even pages with no visible tuples toward lossy and exact pages. After an
> off-list conversation with Andres, it seems that this omission in master
> may not have been intentional.
>
> Once we have only two types of pages to differentiate between (lossy and
> exact [no longer have to care about "has no visible tuples"]), it is
> easy enough to pass a "lossy" boolean paramater to
> table_scan_bitmap_next_block(). I've done this in the attached v4.

Thomas posted a new version of the Streaming Read API [1], so here is a
rebased v5. This should make it easier to review as it can be applied on
top of master.

- Melanie

[1] https://www.postgresql.org/message-id/CA%2BhUKGJtLyxcAEvLhVUhgD4fMQkOu3PDaj8Qb9SR_UsmzgsBpQ%40mail.gmail.com

Attachment Content-Type Size
v5-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch text/x-diff 6.3 KB
v5-0002-BitmapHeapScan-set-can_skip_fetch-later.patch text/x-diff 2.2 KB
v5-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch text/x-diff 14.2 KB
v5-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch text/x-diff 2.2 KB
v5-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch text/x-diff 2.3 KB
v5-0006-EXPLAIN-Bitmap-table-scan-also-count-no-visible-t.patch text/x-diff 3.0 KB
v5-0007-table_scan_bitmap_next_block-returns-lossy-or-exa.patch text/x-diff 4.4 KB
v5-0008-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch text/x-diff 2.9 KB
v5-0009-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch text/x-diff 4.1 KB
v5-0010-Make-table_scan_bitmap_next_block-async-friendly.patch text/x-diff 20.1 KB
v5-0011-Hard-code-TBMIterateResult-offsets-array-size.patch text/x-diff 5.3 KB
v5-0012-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch text/x-diff 19.4 KB
v5-0013-Streaming-Read-API.patch text/x-diff 56.9 KB
v5-0014-BitmapHeapScan-uses-streaming-read-API.patch text/x-diff 28.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message vignesh C 2024-02-27 14:26:55 Re: Can we include capturing logs of pgdata/pg_upgrade_output.d/*/log in buildfarm
Previous Message Peter Geoghegan 2024-02-27 14:22:22 Re: abi-compliance-checker