Re: PostgreSQL 12 service failing in Ubuntu 20.04 after a few hours

I just added postgres to the sudoers list, nothing, same issue (slightly different journalctl logs)

This is a machine in the cloud, I can’t disconnect it.

And yes the ps looks like this precisely when I do a fresh restart. I kill all postgres processes and restart:

christan(at)vultr:~$ for i in `ps -ef | grep postgres | awk '{print $2}'`; do sudo kill $i; done
christan(at)vultr:~$ sudo systemctl restart postgresql

Then this is the output of me ps:

christan(at)vultr:~$ ps -ef | grep postgres
postgres 3504795 1 0 05:51 ? 00:00:00 /usr/lib/postgresql/12/bin/postgres -D /var/lib/postgresql/12/main -c config_file=/etc/postgresql/12/main/postgresql.conf
postgres 3504797 3504795 0 05:51 ? 00:00:00 postgres: 12/main: checkpointer
postgres 3504798 3504795 0 05:51 ? 00:00:00 postgres: 12/main: background writer
postgres 3504799 3504795 0 05:51 ? 00:00:00 postgres: 12/main: walwriter
postgres 3504800 3504795 0 05:51 ? 00:00:00 postgres: 12/main: autovacuum launcher
postgres 3504801 3504795 0 05:51 ? 00:00:00 postgres: 12/main: stats collector
postgres 3504802 3504795 0 05:51 ? 00:00:00 postgres: 12/main: logical replication launcher
christan 3504902 3504620 0 05:59 pts/0 00:00:00 grep --color=auto postgres

After a couple of hours, all these go away and I just see that weird ps output… :(

Then this is what I see in journalctl:

Jan 01 21:36:03 vultr.guest systemd[1]: Starting PostgreSQL Cluster 12-main...
Jan 01 21:36:06 vultr.guest systemd[1]: Started PostgreSQL Cluster 12-main.
Jan 01 22:58:44 vultr.guest sudo[3472381]: postgres : TTY=unknown ; PWD=/var/lib/postgresql/12/main ; USER=root ; COMMAND=/usr/sbin/sysctl kernel.nmi_watchdog=0
Jan 01 22:58:44 vultr.guest sudo[3472381]: pam_unix(sudo:session): session opened for user root by (uid=0)
Jan 01 22:58:44 vultr.guest sudo[3472381]: pam_unix(sudo:session): session closed for user root
Jan 01 22:58:57 vultr.guest crontab[3473870]: (postgres) LIST (postgres)
Jan 01 22:58:57 vultr.guest crontab[3473873]: (postgres) LIST (postgres)
…
…
…
Jan 01 22:58:57 vultr.guest crontab[3474021]: (postgres) LIST (postgres)
Jan 01 22:58:57 vultr.guest crontab[3474024]: (postgres) LIST (postgres)
Jan 01 22:58:57 vultr.guest crontab[3474026]: (postgres) REPLACE (postgres)
Jan 01 22:58:57 vultr.guest postgresql(at)12-main[3474027]: Cluster is not running.
Jan 01 22:58:57 vultr.guest systemd[1]: postgresql(at)12-main(dot)service: Control process exited, code=exited, status=2/INVALIDARGUMENT
Jan 01 22:58:57 vultr.guest systemd[1]: postgresql(at)12-main(dot)service: Failed with result 'exit-code'.

I may just install the same way into a dedicated AWS instance, just to see if I’m getting the same behavior there...

> On 1 Jan 2023, at 11:50 PM, Rob Sargent <robjsargent(at)gmail(dot)com> wrote:
>
> On 1/1/23 14:48, Adrian Klaver wrote:
>> On 1/1/23 13:11, Antonis Christodoulou wrote:
>>> Hello Adrian,
>>>
>>> No it’s not open, but the database itself has very simple credentials (I am just starting with PostgreSQL). What’s weird about the logs?
>>
>> Not the logs the ps output. I would expect to see something like:
>>
>> ps -ef | grep postgres
>> postgres 395382 1 0 2022 ? 00:03:31 /usr/lib/postgresql/14/bin/postgres -D /var/lib/postgresql/14/main -c config_file=/etc/postgresql/14/main/postgresql.conf
>> postgres 395384 395382 0 2022 ? 00:00:01 postgres: 14/main: checkpointer
>> postgres 395385 395382 0 2022 ? 00:00:26 postgres: 14/main: background writer
>> postgres 395386 395382 0 2022 ? 00:00:26 postgres: 14/main: walwriter
>> postgres 395387 395382 0 2022 ? 00:01:45 postgres: 14/main: autovacuum launcher
>> postgres 395388 395382 0 2022 ? 00:00:05 postgres: 14/main: archiver last was 0000000100000003000000B9.00000028.backup
>> postgres 395389 395382 0 2022 ? 00:01:37 postgres: 14/main: stats collector
>> postgres 395390 395382 0 2022 ? 00:02:24 postgres: 14/main: pg_cron launcher
>> postgres 395391 395382 0 2022 ? 00:00:01 postgres: 14/main: logical replication launcher
>>
>> This:
>>
>> postgres 3342383 1 0 2022 ? 00:00:00 FzXlkULu
>> postgres 3344758 1 99 2022 ? 3-14:39:11 OElid7Dp
>> postgres 3419125 1 18 13:57 ? 01:17:03 tracepath
>>
>> just does not look right.
>>
>>
> At the very least disconnect that machine from the internet. Completely.
>