Re: copy.c allocation constant

On Wed, Jan 24, 2018 at 1:43 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
> Indeed. Don't think RAW_BUF_SIZE is quite big enough for that on most
> platforms though. From man mallopt:
> Balancing these factors leads to a default setting of 128*1024 for the M_MMAP_THRESHOLD parameter.
> Additionally, even when malloc() chooses to use mmap() to back an
> allocation, it'll still needs a header to know the size of the
> allocation and such. So exactly using a size of a multiple of 4KB will
> still leave you with wasted space. Due to the latter I can't see it
> mattering whether or not we add +1 to a power-of-two size.

Well, it depends on how it works. dsa_allocate, for example, never
adds a header to the size of the allocation. Allocations < 8kB are
bucketed by size class and stored in superblocks carved up into
equal-sized chunks. Allocations > 8kB are rounded to a multiple of
the 4kB page size and we grab that many consecutive free pages. I
didn't make those behaviors up; I copied them from elsewhere. Some
other allocator I read about did small-medium-large allocations: large
with mmap(), medium with multiples of the page size, small with
closely-spaced size classes.

It doesn't seem like a particularly good idea to take a 64kB+1 byte
allocation, stick a header on it, and pack it tightly up against other
allocations on both sides. Seems like that could lead to
fragmentation problems. Is that really what it does?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company