Understanding was terminated by signal 9: Killed

From: Wells Oliver <wells(dot)oliver(at)gmail(dot)com>
To: pgsql-admin <pgsql-admin(at)postgresql(dot)org>
Subject: Understanding was terminated by signal 9: Killed
Date: 2021-10-19 21:42:15
Message-ID: CAOC+FBWu83+5aiwgzGig38xVQzrWKP1obytTT=P=2r3UgQFiug@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

In an RDS instance with 16GB RAM, I ran a long query which started by
setting temp_buffers to 16GB, so I think I plum ran out of memory, but can
anyone point me in a different direction if the following log messages
indicate something else is awry?

2021-10-19 21:01:41 UTC::@:[24755]:LOG: checkpoint complete: wrote 110375
buffers (22.3%); 0 WAL file(s) added, 0 removed, 14 recycled; write=269.809
s, sync=0.208 s, total=270.089 s; sync files=30, longest=0.200 s,
average=0.007 s; distance=1020117 kB, estimate=1020117 kB
2021-10-19 21:07:11 UTC::@:[24755]:LOG: checkpoint starting: time
2021-10-19 21:10:37 UTC::@:[24752]:LOG: server process (PID 25813) was
terminated by signal 9: Killed
2021-10-19 21:10:37 UTC::@:[24752]:DETAIL: Failed process was running:
refresh materialized view concurrently stats.vwm_d with data;
2021-10-19 21:10:37 UTC::@:[24752]:LOG: terminating any other active
server processes
2021-10-19 21:10:37 UTC::@:[24758]:WARNING: terminating connection because
of crash of another server process
2021-10-19 21:10:37 UTC::@:[24758]:DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.
2021-10-19 21:10:37 UTC::@:[24758]:HINT: In a moment you should be able to
reconnect to the database and repeat your command.
2021-10-19 21:10:37 UTC::@:[24752]:LOG: archiver process (PID 24759)
exited with exit code 2
2021-10-19 21:10:37 UTC::@:[24752]:LOG: all server processes terminated;
reinitializing
2021-10-19 21:10:37 UTC::@:[23697]:LOG: database system was interrupted;
last known up at 2021-10-19 21:01:41 UTC
2021-10-19 21:10:37 UTC::@:[23697]:LOG: database system was not properly
shut down; automatic recovery in progress
2021-10-19 21:10:37 UTC::@:[23697]:LOG: redo starts at 611/C8668F30
2021-10-19 21:10:42 UTC::@:[23697]:LOG: invalid record length at 612/310:
wanted 24, got 0
2021-10-19 21:10:42 UTC::@:[23697]:LOG: redo done at 612/2D8
2021-10-19 21:10:42 UTC::@:[23697]:LOG: checkpoint starting:
end-of-recovery immediate
2021-10-19 21:10:45 UTC:127.0.0.1(28416):rdsadmin(at)rdsadmin:[23907]:FATAL:
the database system is in recovery mode
2021-10-19 21:10:45 UTC:127.0.0.1(28418):rdsadmin(at)rdsadmin:[23908]:FATAL:
the database system is in recovery mode
2021-10-19 21:10:46 UTC::@:[23697]:LOG: checkpoint complete: wrote 88487
buffers (17.8%); 0 WAL file(s) added, 0 removed, 2 recycled; write=3.986 s,
sync=0.015 s, total=4.015 s; sync files=40, longest=0.008 s, average=0.001
s; distance=910940 kB, estimate=910940 kB
2021-10-19 21:10:46 UTC::@:[24752]:LOG: database system is ready to accept
connections
2021-10-19 21:15:46 UTC::@:[23909]:LOG: checkpoint starting: time
2021-10-19 21:15:46 UTC::@:[23909]:LOG: checkpoint complete: wrote 3
buffers (0.0%); 0 WAL file(s) added, 1 removed, 0 recycled; write=0.366 s,
sync=0.005 s, total=0.383 s; sync files=2, longest=0.003 s, average=0.003
s; distance=5 kB, estimate=5 kB
2021-10-19 21:20:47 UTC::@:[23909]:LOG: checkpoint starting: time
2021-10-19 21:20:52 UTC::@:[23909]:LOG: checkpoint complete: wrote 51
buffers (0.0%); 0 WAL file(s) added, 1 removed, 0 recycled; write=4.987 s,
sync=0.004 s, total=5.004 s; sync files=29, longest=0.004 s, average=0.001
s; distance=65682 kB, estimate=65682 kB
2021-10-19 21:25:47 UTC::@:[23909]:LOG: checkpoint starting: time
2021-10-19 21:25:51 UTC::@:[23909]:LOG: checkpoint complete: wrote 47
buffers (0.0%); 0 WAL file(s) added, 1 removed, 0 recycled; write=4.587 s,
sync=0.011 s, total=4.630 s; sync files=27, longest=0.011 s, average=0.001
s; distance=65528 kB, estimate=65667 kB
2021-10-19 21:29:44 UTC::@:[23909]:LOG: checkpoint starting: wal

--
Wells Oliver
wells(dot)oliver(at)gmail(dot)com <wellsoliver(at)gmail(dot)com>

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Tom Lane 2021-10-19 22:01:48 Re: Understanding was terminated by signal 9: Killed
Previous Message Vijaykumar Jain 2021-10-19 17:17:23 Re: Renaming 800GB size table in postgresql