Re: Posix Shared Mem patch

On Thu, Jun 28, 2012 at 1:43 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> On Thu, Jun 28, 2012 at 7:27 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>>> On Thursday, June 28, 2012 07:19:46 PM Magnus Hagander wrote:
>>>> What happens if you mlock() it into memory - does that fail quickly?
>>>> Is that not something we might want to do *anyway*?
>
>>> You normally can only mlock() mminor amounts of memory without changing
>>> settings. Requiring to change that setting (aside that mlocking would be a bad
>>> idea imo) would run contrary to the point of the patch, wouldn't it? ;)
>
>> It would. I wasn't aware of that limitation :)
>
> The OSX man page says that mlock should give EAGAIN for a permissions
> failure (ie, exceeding the rlimit) but
>
>     [ENOMEM]           Some portion of the indicated address range is not
>                        allocated.  There was an error faulting/mapping a
>                        page.
>
> It might be helpful to try mlock (if available, which it isn't
> everywhere) and complain about ENOMEM but not other errors.  If course,
> if the kernel checks rlimit first, we won't learn anything ...

I tried this. At least on my fairly vanilla MacOS X desktop, an mlock
for a larger amount of memory than was conveniently on hand (4GB, on a
4GB box) neither succeeded nor failed in a timely fashion but instead
progressively hung the machine, apparently trying to progressively
push every available page out to swap. After 5 minutes or so I could
no longer move the mouse. After about 20 minutes I gave up and hit
the reset button. So there's apparently no value to this as a
diagnostic tool, at least on this platform.

> I think it *would* be a good idea to mlock if we could.  Setting shmem
> large enough that it swaps has always been horrible for performance,
> and in sysv-land there's no way to prevent that.  But we can't error
> out on permissions failure.

I wouldn't mind having an option, but I think there'd have to be a way
to turn it off for people trying to cram as many lightly-used VMs as
possible onto a single server.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company