Re: dynamically allocating chunks from shared memory

On Mon, Aug 9, 2010 at 4:18 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> On Mon, Aug 9, 2010 at 3:20 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> It's not portable.  That's exactly what we were looking into back when.
>
>> Uggh, that sucks.  Can you provide any more details?
>
> You don't really have to go further than consulting the relevant
> standards, eg SUS says at
> http://www.opengroup.org/onlinepubs/007908799/xsh/mmap.html
>
>        If the size of the mapped file changes after the call to mmap() as a
>        result of some other operation on the mapped file, the effect of
>        references to portions of the mapped region that correspond to added
>        or removed portions of the file is unspecified.
>
> Particular implementations might cope with such cases in useful ways, or
> then again they might not.

That doesn't seem like a big problem to me. I was assuming we'd need
to remap when the size changed. Also, I was assuming that we were
going to use shms, not files. Take a look at this:

http://www.opengroup.org/onlinepubs/007908799/xsh/shm_open.html -and-
http://www.opengroup.org/onlinepubs/007908799/xsh/ftruncate.html

From the ftruncate page: "If fildes references a shared memory object,
ftruncate() sets the size of the shared memory object to length."

> And even if your platform does, you've set
> an upper limit for the possible segment size in your mmap() call.
>
> Further down the page, SUS also takes pains to point out that you
> probably can't have an unlimited number of mapped regions, so adding
> more mmap'd segments isn't a way out either.

Yeah. I think any approach that is based on allocating new segments
as needed is pretty much DOA. I think the point of this would be to
be able to resize things like shared_buffers on the fly - that is, an
explicit administrator action might trigger a resize-and-remap cycle,
but general system activity would not. The reality is that as
PostgreSQL is used in more and more 24x7 contexts and people put more
and more critical data into it, forced server restarts become more and
more of a problem. IMHO, we really need to do some creative thinking
about how to crank PGC_POSTMASTER GUCs down to PGC_SIGHUP.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company