Re: MaxOffsetNumber for Table AMs

On Fri, 2021-04-30 at 12:04 +0200, Matthias van de Meent wrote:
> Other than that, I believe you've also missed the special offset
> numbers specified in itemptr.h (SpecTokenOffsetNumber and
> MovedPartitionsOffsetNumber). I am not well enough aware of the usage
> of these OffsetNumber values, but those might also be limiting the
> values any tableAM can use for their TIDs.

Yes, thank you. If it is treated specially in a heap tuple, it can't be
a regular TID.

> > It just seems like it's used when scanning heap or index pages for
> > stack-allocated arrays. For a table AM it appears to waste 5 bits.
>
> MaxOffsetNumber is used for postgres' Page layout, of which the
> MaxOffsetNumber is defined as how many item pointers could exist on a
> page, and AFAIK should be used for postgres' Page layout only. No
> thing can or should change that. If any code asserts limitations to
> the ip_posid of table tuples that could also not be tableam tuples,
> then I believe that is probably a mistake in postgres, and that
> should be fixed.

A name that would better fit your definition would be something like
"MaxItemsPerPage".

The name "MaxOffsetNumber" implies that any number past that must be
either invalid or special. But it seems like you are saying that if I
use an offset number of 5000 in my table AM, then that's fine and
should be treated like a normal TID.

> No. The documentation for that function explicitly mentions that
> these item pointers are optimized for storage when using the heap
> tableam, and that that code will be changed once there exist tableAMs
> with different TID / ip_posid constraints (see the comment on lines
> 32 and 33 of that file).

Thank you.

I'm a table AM author, and I'd like to use whatever the real range of
TIDs is. Does that mean it's time to change that code in
ginpostinglist.c now?

> > 1. Keep MaxOffsetNumber as 2048 and fix itemptr_to_uint64().
>
> I believe that this is the right way when there exist tableAMs that
> use those upper 5 bits.

Does that mean we should declare the valid range of offsets to be
between 1 and 0xfffc (inclusive)?

I'm trying to use some mapping now that's somewhat stable so that I
don't have to worry that something will break later, and then require
reindexing all tables with my table AM.

Regards,
Jeff Davis