Re: FATAL: terminating connection due to administrator command

On Thu, Oct 1, 2020 at 2:47 PM Alban Hertroys <
alban(dot)hertroys(at)apollovredestein(dot)com> wrote:

> Hi all,
>
> We're seeing the FATAL error message from the subject pop up in our logs
> at regular intervals, but I haven't been able to pinpoint what is causing
> it. I'm hoping for some insights here.
>
> We run a PostgreSQL 11.9 server on CentOS 7, within a vmware environment:
> PostgreSQL 11.9 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5
> 20150623 (Red Hat 4.8.5-39), 64-bit
>
> The package was installed from the PGDG repository.
>
> I'm not even sure I should be worried, there doesn't appear to be any
> impact on the servers' functioning, but it does say 'FATAL'.
> What we're seeing are lines like these two instances:
>
> 2020-09-30 22:27:56.446 CEST [30659] STATEMENT: select count(*) from
> "dm_b2b"."prlwytzkofskiv1"
> 2020-09-30 22:27:56.446 CEST [30658] FATAL: terminating connection due
> to administrator command
> 2020-09-30 22:27:56.446 CEST [30658] STATEMENT: select count(*) from
> "dm_b2b"."prlwytzkofskiv1"
> 2020-09-30 22:27:56.446 CEST [30657] FATAL: terminating connection due
> to administrator command
> 2020-09-30 22:27:56.446 CEST [30657] STATEMENT: select count(*) from
> "dm_b2b"."prlwytzkofskiv1"
> 2020-09-30 22:27:56.446 CEST [30656] FATAL: terminating connection due
> to administrator command
> 2020-09-30 22:27:56.446 CEST [30656] STATEMENT: select count(*) from
> "dm_b2b"."prlwytzkofskiv1"
> 2020-09-30 22:27:56.446 CEST [30655] FATAL: terminating connection due
> to administrator command
> 2020-09-30 22:27:56.446 CEST [30655] STATEMENT: select count(*) from
> "dm_b2b"."prlwytzkofskiv1"
> 2020-09-30 22:27:56.459 CEST [6482] LOG: background worker "parallel
> worker" (PID 30655) exited with exit code 1
> 2020-09-30 22:27:56.459 CEST [6482] LOG: background worker "parallel
> worker" (PID 30656) exited with exit code 1
> 2020-09-30 22:27:56.459 CEST [6482] LOG: background worker "parallel
> worker" (PID 30657) exited with exit code 1
> 2020-09-30 22:27:56.459 CEST [6482] LOG: background worker "parallel
> worker" (PID 30658) exited with exit code 1
> 2020-09-30 22:27:56.459 CEST [6482] LOG: background worker "parallel
> worker" (PID 30659) exited with exit code 1
> 2020-09-30 22:43:08.459 CEST [8055] 172.30.2.25 selfservice_prd ERROR:
> schema "somethingelse" does not exist at character 71
>
> I am guessing that 6 background workers are started, 1 worker had the
result and hence killing the other 5 workers. Maybe, some more pg experts
can comment. Anyway, explain of your query helps.

Regards,
Seenu.

Apparently, something is sending SIGTERM to our pg processes. I know that
> I'm not doing that, certainly not at those hours, and I'm the one who set
> up this system and am the only DBA of it.
>
> Advice I found on the Internet is to use systemtap with some tap-script,
> but the scripts that I found just displayed the PID's of processes without
> telling me their names, which I didn't find all that useful in figuring out
> who was responsible, so I made an attempt (I have no experience with stap)
> at modifying it to print process names of signal sender and target:
>
>
> *The information contained in this e-mail is intended solely for the use
> of the individual or entity to whom it is addressed. If you are not the
> intended recipient, you are hereby notified that any disclosure, copying,
> distribution or action in relation to the contents of this information is
> strictly prohibited and may be unlawful and request you to delete this
> message and any attachments and advise the sender by return e-mail. The
> confidentiality of this message is not warranted. Apollo Vredestein and its
> subsidiaries rule out any and every liability resulting from this or any
> other electronic transmission*
> Please consider the environment before printing this e-mail
>