Re: FATAL: semctl(15073290, 4, SETVAL, 0) failed: Invalid argument

On 03/20/2018 11:57 AM, JotaComm wrote:
> ​Hello,
>
> Today I found this message in my Postgres log:
>
> FATAL:  semctl(15073290, 4, SETVAL, 0) failed: Invalid argument
> user=,db=,app=,host= LOG:  server process (PID 30741) exited with exit
> code 1
> user=,db=,app=,host= LOG:  terminating any other active server processes
> WARNING:  terminating connection because of crash of another server process
> DETAIL:  The postmaster has commanded this server process to roll back
> the current transaction and exit, because another server process exited
> abnormally and possibly corrupted shared memory.
> HINT:  In a moment you should be able to reconnect to the database and
> repeat your command.
>
> Postgres: PostgreSQL 9.6.3 on x86_64-pc-linux-gnu, compiled by gcc
> (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609, 64-bit
>
> Linux: Linux INVST-APP-01A 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18
> 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
>
> FATAL:  semctl(15073290, 4, SETVAL, 0) failed: Invalid argument
> user=,db=,app=,host= LOG:  server process (PID 30741) exited with exit
> code 1
> user=,db=,app=,host= LOG:  terminating any other active server processes
> WARNING:  terminating connection because of crash of another server process
> DETAIL:  The postmaster has commanded this server process to roll back
> the current transaction and exit, because another server process exited
> abnormally and possibly corrupted shared memory.
> HINT:  In a moment you should be able to reconnect to the database and
> repeat your command.
>
> Any suggestions to discovery about this behavior?

?:
https://www.postgresql.org/docs/10/static/kernel-resources.html
"
18.4.2. systemd RemoveIPC

If systemd is in use, some care must be taken that IPC resources (shared
memory and semaphores) are not prematurely removed by the operating
system. This is especially of concern when installing PostgreSQL from
source. Users of distribution packages of PostgreSQL are less likely to
be affected, as the postgres user is then normally created as a system user.

The setting RemoveIPC in logind.conf controls whether IPC objects are
removed when a user fully logs out. System users are exempt. This
setting defaults to on in stock systemd, but some operating system
distributions default it to off.

A typical observed effect when this setting is on is that the semaphore
objects used by a PostgreSQL server are removed at apparently random
times, leading to the server crashing with log messages like

LOG: semctl(1234567890, 0, IPC_RMID, ...) failed: Invalid argument

Different types of IPC objects (shared memory vs. semaphores, System V
vs. POSIX) are treated slightly differently by systemd, so one might
observe that some IPC resources are not removed in the same way as
others. But it is not advisable to rely on these subtle differences.

A “user logging out” might happen as part of a maintenance job or
manually when an administrator logs in as the postgres user or something
similar, so it is hard to prevent in general.

What is a “system user” is determined at systemd compile time from the
SYS_UID_MAX setting in /etc/login.defs.

Packaging and deployment scripts should be careful to create the
postgres user as a system user by using useradd -r, adduser --system, or
equivalent.

Alternatively, if the user account was created incorrectly or cannot be
changed, it is recommended to set

RemoveIPC=no

in /etc/systemd/logind.conf or another appropriate configuration file.
Caution

At least one of these two things has to be ensured, or the PostgreSQL
server will be very unreliable.
"

>
> Thanks a lot.
>
> Best regards
>
> --
> JotaComm
> http://jotacomm.wordpress.com

--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com