Re: Reducing the chunk header sizes on all memory context types

Hi,

On 2022-08-09 15:21:57 -0400, Tom Lane wrote:
> Andres Freund <andres(at)anarazel(dot)de> writes:
> > I think it's fine, given that we can change this at any time, but it's
> > probably worth to explicitly agree that this will for now restrict us to 8
> > context methods?
>
> Do we really need it to be that tight? I know we only have 3 methods today,
> but 8 doesn't seem that far away. If there were six bits reserved for
> this I'd be happier.

We only have so many bits available, so that'd have to come from some other
resource. The current division is:

+ * 1. 3-bits to indicate the MemoryContextMethodID
+ * 2. 1-bit to indicate if the chunk is externally managed (see below)
+ * 3. 30-bits for the amount of memory which was reserved for the chunk
+ * 4. 30-bits for the number of bytes that must be subtracted from the chunk
+ * to obtain the address of the block that the chunk is stored on.

I suspect we could reduce 3) here a bit, which I think would end up with slab
context's max chunkSize shrinking further. Which should still be fine.

But also, we could defer that to later, this is a limit that we can easily
change.

> >> # We also add a restriction that block sizes for all 3 of the memory
> >> # allocators cannot be 1GB or larger. We would be unable to store the
> >> # number of bytes that the block is offset from the chunk stored beyond this
> >> #1GB boundary on any block that was larger than 1GB.
>
> Losing MemoryContextAllocHuge would be very bad, so I assume this comment
> is not telling the full truth.

It's just talking about chunked allocation (except for slab, which doesn't
have anything else, but as David pointed out, it makes no sense to use slab
for that large allocations). I.e. it's the max you can pass to
AllocSetContextCreate()'s and GenerationContextCreate()'s maxBlockSize, and to
SlabContextCreate()'s chunkSize. I don't think we have any code that
currently sets a bigger limit than 8MB.

Greetings,

Andres Freund