Re: Back-patch use of unnamed POSIX semaphores for Linux?

On Thu, Dec 8, 2016 at 12:40 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Ah-hah, thanks for the insight. I can now reproduce it, and I confirm
> that aside from removing the semaphores, our POSIX shmem segment(s)
> are removed from /dev/shm. They presumably still are attached to whatever
> processes have them mapped already, but this behavior is going to break
> DSM usage in any case. (The SysV shm segment does survive, presumably
> because systemd notices its positive attach count.)

Make sense. Actually, it would be fairly unlucky for the DSM thing to
cause a failure for parallel query as it exists today, because there's
only about 4ms between when the segment is created and when all
backends attached. But DSA - especially if we use it for anything
long-lived - will surely break.

> So I now agree that getting out from under SysV semaphores isn't going to
> fix our problems with systemd ... at least, not unless we migrate *to*
> not away from SysV shared memory for DSM. Even then, we'd have to be
> careful that standard usage patterns keep every DSM segment continually
> attached to at least one process. Dunno how practical that is. And it
> blows chunks in the goal of not being constrained by SHMMAX.

dynamic_shared_memory_type = sysv is already supported, but it's not
the default precisely because of SHMMAX. Keeping every DSM segment
attached to at least one process is normal for parallel query as it
exists today, but tough for any application that "pins" segments. On
Windows, we do that: pinning a segment actually causes a user backend
to reach inside the postmaster's address space to open a file
descriptor. Why Microsoft thought that was something a process should
be able to do to another process I couldn't say. But on Linux where
segments don't go away on last close, and where such frightening APIs
don't exist, there's no guarantee that a segment will always be open
somewhere.

Hey, I have an idea. Let's switch from processes to threads, and then
shared memory - including the dynamic kind - can be implemented using
malloc().

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company