Re: dynamically allocating chunks from shared memory

On Mon, Aug 9, 2010 at 11:02 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> I am not sure threads would greatly help us.  The major problem is that
> all of our our structures are currently contiguous in memory for quick
> access.  I don't see how threading would help with that.  We could use
> realloc(), but we can do the same in shared memory if we had a chunk
> infrastructure, though concurrent access to that memory would hurt us in
> either threads or shared memory.
>
> Fundamentally, recreating the libc memory allocation routines is not
> that hard.  (Everyone has to detach from the shared memory segment, but
> they have to stop using it too, so it doesn't seem that hard.)

I actually don't think that's true. The advantage (and disadvantage)
of using threads is that everything runs in one address space. So you
just allocate more memory and everyone immediately sees it. In a
process environment, that's not the case: to expand or shrink the size
of the shared memory arena, everyone needs to explicitly change their
own mapping.

So imagine that thread-or-process A allocates allocates a new chunk of
memory and then writes a pointer to the new chunk in a previously
allocated section of memory. Thread-or-process B then follows the
pointer. In a threaded model, this is guaranteed to be safe. In a
process model, it's not: A might have enlarged the shared memory
mapping while B has not yet done so. So I think in our model any sort
of change to the shared memory segment is going to require extremely
careful gymnastics, and be pretty expensive.

I don't care to take a position in the religious war over threads vs.
processes, but I do think threads simplify the handling of this
particular case.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company