Re: MaxOffsetNumber for Table AMs

On Mon, May 3, 2021 at 10:22 AM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> I don't really think so, or at least I don't see a reason why it
> should. As things stand today, I don't think it's possible for a table
> AM author to make any other choice than to assume that their TIDs have
> to look and work like heap TIDs; that is, there had better be a block
> number portion and an item number portion, and the item number had
> better be smaller than MaxOffsetNumber, and if you want bitmap scans
> to run reasonably quickly, the block number had also better correspond
> to physical locality to some degree. It's not clear to me how exactly
> someone would go about fixing all of that, but I think it would be
> great if they did. Even if that person wanted to assume for purposes
> of their own patch that fixed-width, integer-like TIDs are the only
> thing we care about, that would be fine with me. Getting to a point
> where the available 48 bits can be used in whatever way the table AM
> author wants is clearly better than what we have now.

I don't think it's much good to just do that. You probably need a full
64-bits for something like a column store. But that's all you need.

> Now I'm personally of the opinion that we shouldn't be content to stop
> there, but so what? I'm not insisting that Jeff or anyone else has to
> work on this problem, or that they have to fix more of it rather than
> less. I hope that nobody's going to try to back us into a corner by
> making design decisions that deliberately complicate the possibility
> of future improvements in that area, and that's about it. I don't
> really understand why you think that's unreasonable, or even
> problematic. I can't see that any way in which the assumption that we
> will NEVER want to further generalize the TID concept simplifies
> anything anyone wants to do today.

It creates ambiguity of the kind that deters related improvements. I
for one am not comfortable with (say) working on generalizing TID to
the extent required to facilitate Jeff's work if that obligates me to
make some legalistic and wholly insincere statement about future
improvements to the definition of TID still being quite possible (to
facilitate indirect indexes, or whatever). The truth is that I cannot
possibly know if facilitating Jeff's work in the short term blocks off
other things in the long term -- because I don't actually have a clue
how these other things could ever really be implemented sensible in
any case.

--
Peter Geoghegan