Re: dynamically allocating chunks from shared memory

Hi,

On 07/22/2010 08:31 PM, Alvaro Herrera wrote:
> FWIW I don't think you should be thinking in "replacing imessages with
> SLRU". I rather think you should be thinking in how can you implement
> the imessages API on top of SLRU.

Well, I'm rather comparing SLRU with the dynamic allocator. So far I'm
unconvinced that SLRU would be a better base for imessages than a
dynamic allocator. (And I'm arguing that SLRU should use a dynamic
allocator underneath).

> So as far as the coordinator and
> background worker are concerned, there wouldn't be any difference --
> they keep using the same API they are using today.

Agreed, the imessages API to the upper layer doesn't need to care about
the underlying stuff.

> Also let me repeat my earlier comment about imessages being more similar
> to multixact than to notify. The content of each multixact entry is
> just an arbitrary amount of bytes. If imessages are numbered from a
> monotonically increasing sequence,

Well, there's absolutely no need to serialize imessages. So they don't
currently carry any such number. And opposed to multixact entries, they
are clearly directed at exactly one single consumer. Every consumer has
its own receive queue. Sending messages concurrently to different
recipients may happen completely parallelized, without any (b)locking in
between.

The dynamic allocator is the only part of the chain which might need to
do some locking to protect the shared resource (memory) against
concurrent access. Note, however, that wamalloc (as any modern dynamic
allocator) is parallelized to some extent, i.e. concurrent malloc/free
calls don't necessarily need to block each other.

> it should be possible to use a very
> similar technique, and perhaps you should be able to reduce locking
> requirements as well (write messages with only a shared lock, after
> you've determined and reserved the area you're going to write).

Writing to the message is currently (i.e. imessages-on-dynshmem) done
without *any* kind of lock held. So that would rather increase locking
requirements and lower parallelism, I fear.

Regards

Markus