Re: High CPU usage

On 2022-10-20 15:59, ertan(dot)kucukoglu(at)1nar(dot)com(dot)tr wrote:
> Hello,
>
> I am using PostgreSQL v14.5 on Linux Debian 11.5. I recently observe very
> high CPU usage on my Linux system as below
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
> COMMAND
> 2357756 postgres 20 0 2441032 2,3g 4 S 298,7 67,9 2114:58
> Tspjzj2Z
>
> I could not find any file named Tspjzj2Z on the file system. I could not
> find PID number using below SQL

Hi,

I'm not an expert in PostgreSQL but that looks like a rogue app, if
you're lucky just a miner running as the prostgres user, likely the
result of a postgres RCE exploited successfully... The more worring case
would be a program exfiltrating and/or encrypting the database in a
ransomware attack.

The executable has most likely been removed to hide traces, or cleaned
up automatically from ex. /tmp, however if the process is still running
you should be able to cat the executable, and any other open files,
directly from /proc/<pid>/ (look for exe and fd/*).

I strongly recommend you check other postgres servers you have, make a
copy of any process file found (for later investigation), then isolate
or shutdown these servers and proceed with a proper investigation from a
livecd or revovery OS.

> There is no replication of any kind. This is a single instance server which
> alows certification login only.

Is is even available from the outside world? Else you should likely
audit any internal hosts that could have accessed your postgresql
server. If you have firewall logs looks for unusual connection attempts,
any evidence of scanning, etc.

Hackers will often spend quite some time once inside to gather as much
information as possible before doing any real damage, although if this
is effectively a miner it would be less likely to be that kind of attack
as they would probably not risk getting discovered with something that
will at best make them pennies...

If you see the process having any open database files, it's possible
it's either compressing them to exfiltrate the data or encrypting them,
or both....

Hope this helps...

--
Thomas