Re: Dynamic shared memory areas

On Fri, Aug 19, 2016 at 7:07 PM, Thomas Munro
<thomas(dot)munro(at)enterprisedb(dot)com> wrote:
> I would like to propose a new subsystem called Dynamic Shared [Memory]
> Areas, or "DSA". It provides an object called a "dsa_area" which can
> be used by multiple backends to share data. Under the covers, a
> dsa_area is made up of some number of DSM segments, but it appears to
> client code as a single shared memory heap with a simple allocate/free
> interface. Because the memory is mapped at different addresses in
> different backends, it introduces a kind of sharable relative pointer
> and an operation to convert it to a backend-local pointer.
>
> [...]
>
> [...] It's desirable to allow atomic ops on
> dsa_pointer; I believe Andres Freund plans to make that happen for 64
> bit values on 32 bit systems, but if that turns out to be problematic
> I would want to make dsa_pointer 32 bits on 32 bit systems.

Here's a new version that does that. It provides the type
dsa_pointer_atomic and associated operations, using
PG_HAVE_ATOMIC_U64_SUPPORT to decide which size to use. The choice of
size is overridable at compile time with USE_SMALL_DSA_POINTER.

The other change is that it now creates DSM segments of sizes that
don't get large so fast. V1 would create 1MB, 2MB, 4MB, ... segments
(geometric growth being necessary because we can't have large numbers
of segments, but we want to support large total sizes). V2 creates
segments of size 1, 1, 1, 1, 2, 2, 2, 2, 4, 4, 4, 4, ... according to
the compile time constant DSA_NUM_SEGMENTS_AT_EACH_SIZE. I'm not sure
how to select a good number for this yet and the best answer may
depend on whether you're using small pointers.

This version is rebased against master as of today and doesn't depend
on any other patches.

--
Thomas Munro
http://www.enterprisedb.com