Re: Status of the table access method work

On 05/04/2019 23:25, Andres Freund wrote:
> I think what's in v12 - I don't know of any non-cleanup / bugfix work
> pending for 12 - is a pretty reasonable initial set of features.

Hooray!

> - the (optional) bitmap heap scan API - that's fairly intrinsically
> block based. An AM could just internally subdivide TIDs in a different
> way, but I don't think a bitmap scan like we have would e.g. make a
> lot of sense for an index oriented table without any sort of stable
> tid.

If an AM doesn't implement the bitmap heap scan API, what happens?
Bitmap scans are disabled?

Even if an AM isn't block-oriented, the bitmap heap scan API still makes
sense as long as there's some correlation between TIDs and physical
location. The only really broken thing about that currently is the
prefetching: nodeBitmapHeapScan.c calls PrefetchBuffer() directly with
the TID's block numbers. It would be pretty straightforward to wrap that
in a callback, so that the AM could do something different.

Or move even more of the logic to the AM, so that the AM would get the
whole TIDBitmap in table_beginscan_bm(). It could then implement the
fetching and prefetching as it sees fit.

I don't think it's urgent, though. We can cross that bridge when we get
there, with the first AM that needs that flexibility.

> The most constraining factor for storage, I think, is that currently the
> API relies on ItemPointerData style TIDs in a number of places (i.e. a 6
> byte tuple identifier).

I think 48 bits would be just about enough, but it's even more limited
than you might at the moment. There are a few places that assume that
the offsetnumber <= MaxHeapTuplesPerPage. See ginpostinglist.c, and
MAX_TUPLES_PER_PAGE in tidbitmap.c. Also, offsetnumber can't be 0,
because that makes the ItemPointer invalid, which is inconvenient if you
tried to use ItemPointer as just an arbitrary 48-bit integer.

- Heikki