From: | Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com> |
---|---|
To: | pgsql-hackers(at)postgreSQL(dot)org |
Subject: | PATCH: two slab-like memory allocators |
Date: | 2016-08-02 15:44:35 |
Message-ID: | d15dff83-0b37-28ed-0809-95a5cc7292ad@2ndquadrant.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi,
Back in the bug #14231 thread [1], dealing with performance issues in
reorderbuffer due to excessive number of expensive free() calls, I've
proposed to resolve that by a custom slab-like memory allocator,
suitable for fixed-size allocations. I'd like to put this into the next
CF, and it's probably too invasive change to count as a bugfix anyway.
[1]
https://www.postgresql.org/message-id/flat/20160706185502.1426.28143%40wrigleys.postgresql.org
This patch actually includes two new memory allocators (not one). Very
brief summary (for more detailed explanation of the ideas, see comments
at the beginning of slab.c and genslab.c):
Slab
----
* suitable for fixed-length allocations (other pallocs fail)
* much simpler than AllocSet (no global freelist management etc.)
* free space is tracked per block (using a simple bitmap)
* which allows freeing the block once all chunks are freed (AllocSet
will hold the memory forever, in the hope of reusing it)
GenSlab
-------
* suitable for non-fixed-length allocations, but with chunks of mostly
the same size (initially unknown, the context will tune itself)
* a combination AllocSet and Slab (or a sequence of Slab allocators)
* the goal is to do most allocations in Slab context
* there's always a single 'current' Slab context, and every now and and
then it's replaced with a new generation (with the chunk size computed
from recent requests)
* the AllocSet context is used for chunks too large for current Slab
So none of this is meant as a universal replacement of AllocSet, but in
the suitable cases the results seem really promising. For example for
the simple test query in [1], the performance improvement is this:
N | master | patched
-----------------------------
10000 | 100ms | 100ms
50000 | 15000ms | 350ms
100000 | 146000ms | 700ms
200000 | ? | 1400ms
That's a fairly significant improvement, and the submitted version of
the patches should perform even better (~2x, IIRC).
There's a bunch of TODOs - e.g. handling of realloc() calls in the
GenSlab, and probably things I haven't thought about.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachment | Content-Type | Size |
---|---|---|
0001-simple-slab-allocator-fixed-size-allocations.patch | binary/octet-stream | 35.3 KB |
0002-generational-slab-auto-tuning-allocator.patch | binary/octet-stream | 16.2 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Andrew Gierth | 2016-08-02 16:43:42 | Re: Wanting to learn about pgsql design decision |
Previous Message | Simon Riggs | 2016-08-02 15:43:32 | Re: Why we lost Uber as a user |