Re: [HACKERS] HASH_CHUNK_SIZE vs malloc rounding

Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com> writes:
> On Tue, Nov 29, 2016 at 6:27 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> We could imagine providing an mmgr API function along the lines of "adjust
>> this request size to the nearest thing that can be allocated efficiently".
>> That would avoid the need for callers to know about aset.c overhead
>> explicitly. I'm not sure how it could deal with platform-specific malloc
>> vagaries though :-(

> Someone pointed out to me off-list that jemalloc's next size class
> after 32KB is in fact 40KB by default[1]. So PostgreSQL uses 25% more
> memory for hash joins than it thinks it does on some platforms. Ouch.

> It doesn't seem that crazy to expose aset.c's overhead size to client
> code does it? Most client code wouldn't care, but things that are
> doing something closer to memory allocator work themselves like
> dense_alloc could care. It could deal with its own overhead itself,
> and subtract aset.c's overhead using a macro.

Seeing that we now have several allocators with different overheads,
I think that exposing this directly to clients is exactly what not to do.
I still like the idea I sketched above of a context-type-specific function
to adjust a request size to something that's efficient.

But there's still the question of how do we know what's an efficient-sized
malloc request. Is there good reason to suppose that powers of 2 are OK?

regards, tom lane