Re: dynamically allocating chunks from shared memory

On Mon, Aug 9, 2010 at 2:50 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Robert Haas wrote:
>> On Mon, Aug 9, 2010 at 2:33 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> >> > Let me be more concrete. ?Suppose you are using threads, and you want to
>> >> > increase your shared memory from 20MB to 30MB. ?How do you do that? ?If
>> >> > you want it contiguous, you have to use realloc, which might move the
>> >> > pointer. ?If you allocate another 10MB chunk, you then have shared
>> >> > memory fragments, which is the same as adding another shared memory
>> >> > segment.
>> >>
>> >> You probably wouldn't do either of those things. ?You'd just allocate
>> >> small chunks here and there for whatever you need them for.
>> >
>> > Well, then we do that with shared memory then --- my point is that it is
>> > the same problem with threads or processes.
>>
>> Well, I think your point is wrong, then.  :-)
>>
>> It's not the same at all.  If you have a bunch of threads in one
>> address space, "shared" memory is really just process-local.  You can
>> grow the total amount of allocated space just by calling malloc().
>> With our architecture, you can't.
>
> You effectively have to add infrastructure to add/remove shared memory
> segments to match memory requests.  It is another step, but it is the
> same behavior.

That would be one way to tackle the problem, but there are
difficulties. If we just created new shared memory segments at need,
we might end up with a lot of shared memory segments. I suspect that
would get complicated and present many management difficulties - which
is why I'm so far of the opinion that we should try to architect the
system to avoid the need for this functionality. I don't think it's
going to be too easy to provide, short of (as Tom says) moving to the
MySQL model of many threads working in a single process.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company