Re: dynamically allocating chunks from shared memory

On 07/21/2010 01:52 AM, Robert Haas wrote:
> On Tue, Jul 20, 2010 at 5:46 PM, Alvaro Herrera
> <alvherre(at)commandprompt(dot)com> wrote:
>> I guess what Robert is saying is that you don't need shmem to pass
>> messages around. The new LISTEN implementation was just an example.
>> imessages aren't supposed to use it directly. Rather, the idea is to
>> store the messages in a new SLRU area. Thus you don't need to mess with
>> dynamically allocating shmem at all.

Okay, so I just need to grok the SLRU stuff. Thanks for clarifying.

Note that I sort of /want/ to mess with shared memory. It's what I know
how to deal with. It's how threaded programs work as well. Ya know,
locks, conditional variables, mutexes, all those nice thing that allow
you to shoot your foot so terribly nicely... Oh, well...

>> I think it should be rather straightforward. There would be a unique
>> append-point;

Unique append-point? Sounds like what I had before. That'd be a step
backwards, compared to the per-backend queue and an allocator that
hopefully scales well with the amount of CPU cores.

>> each process desiring to send a new message to another
>> backend would add a new message at that point. There would be one read
>> pointer per backend, and it would be advanced as messages are consumed.
>> Old segments could be trimmed as backends advance their read pointer,
>> similar to how sinval queue is handled.

That leads to pretty nasty fragmentation. A dynamic allocator should do
much better in that regard. (Wamalloc certainly does).

> If the messages are mostly unicast, it might be nice if to contrive a
> method whereby backends didn't need to explicitly advance over
> messages destined only for other backends. Like maybe allocate a
> small, fixed amount of shared memory sufficient for two "pointers"
> into the SLRU area per backend, and then use the SLRU to store each
> message with a header indicating where the next message is to be
> found.

That's pretty much how imessages currently work. A single list of
messages queued per backend.

> For each backend, you store one pointer to the first queued
> message and one pointer to the last queued message. New messages can
> be added by making the current last message point to a newly added
> message and updating the last message pointer for that backend. You'd
> need to think about the locking and reference counting carefully to
> make sure you eventually freed up unused pages, but it seems like it
> might be doable.

I've just read through slru.c, but still don't have a clue how it could
replace a dynamic allocator.

At the moment, the creator of an imessage allocs memory, copies the
payload there and then activates the message by appending it to the
recipient's queue. Upon getting signaled, the recipient consumes the
message by removing it from the queue and is obliged to release the
memory the messages occupies after having processed it. Simple and
straight forward, IMO.

The queue addition and removal is clear. But how would I do the
alloc/free part with SLRU? Its blocks are fixed size (BLCKSZ) and the
API with ReadPage and WritePage is rather unlike a pair of alloc() and
free().

> One big advantage of attacking the problem with an SLRU is that
> there's no fixed upper limit on the amount of data that can be
> enqueued at any given time. You can spill to disk or whatever as
> needed (although hopefully you won't normally do so, for performance
> reasons).

Yes, imessages shouldn't ever be spilled to disk. There naturally must
be an upper limit for them. (Be it total available memory, as for
threaded things or a given and size-constrained pool, as is the case for
dynshmem).

To me it rather sounds like SLRU is a candidate for using dynamically
allocated shared memory underneath, instead of allocating a fixed amount
of slots in advance. That would allow more efficient use of shared
memory. (Given SLRU's ability to spill to disk, it could even be used to
'balance' out anomalies to some extent).

Regards

Markus Wanner