How to debug "background worker "parallel worker" (PID 32152) was terminated by signal 11: Segmentation fault"

From: Radu Radutiu <rradutiu(at)gmail(dot)com>
To: pgsql-general <pgsql-general(at)postgresql(dot)org>
Subject: How to debug "background worker "parallel worker" (PID 32152) was terminated by signal 11: Segmentation fault"
Date: 2020-04-27 08:45:26
Message-ID: CAG4TxriCbtpjtxuRcUm=9Jc-qG1069_36ASH2R+SnCAta2P5KQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi,

Can you guide me how to debug postgresql crash?
I have a development VM (kvm) running Centos 8.1(up to date with all
patches, kernel 4.18.0-147.8.1.el8_1.x86_64,
postgresql12-server-12.2-2PGDG.rhel8.x86_64 ) with a very large partitioned
table (few hundred GBs) . Creating an index on partition tables crashed the
postgresql instance. I have in the log:

2020-04-26 17:35:50.065 EEST [8385] LOG: background worker "parallel
worker" (PID 32152) was terminated by signal 11: Segmentation fault
2020-04-26 17:35:50.065 EEST [8385] DETAIL: Failed process was running:
CREATE INDEX IDX_instrid
ON ach.achiteminfo USING btree
(instrid);
2020-04-26 17:35:50.065 EEST [8385] LOG: terminating any other active
server processes
2020-04-26 17:35:50.065 EEST [32134] WARNING: terminating connection
because of crash of another server process
2020-04-26 17:35:50.065 EEST [32134] DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.
2020-04-26 17:35:50.065 EEST [32134] HINT: In a moment you should be able
to reconnect to the database and repeat your command.
2020-04-26 17:35:50.065 EEST [32146] WARNING: terminating connection
because of crash of another server process
2020-04-26 17:35:50.065 EEST [32146] DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.
2020-04-26 17:35:50.065 EEST [32146] HINT: In a moment you should be able
to reconnect to the database and repeat your command.
2020-04-26 17:35:50.065 EEST [32126] WARNING: terminating connection
because of crash of another server process
2020-04-26 17:35:50.065 EEST [32126] DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.
2020-04-26 17:35:50.065 EEST [32126] HINT: In a moment you should be able
to reconnect to the database and repeat your command.
2020-04-26 17:35:50.066 EEST [32130] WARNING: terminating connection
because of crash of another server process
2020-04-26 17:35:50.066 EEST [32130] DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.
2020-04-26 17:35:50.066 EEST [32130] HINT: In a moment you should be able
to reconnect to the database and repeat your command.
2020-04-26 17:35:50.066 EEST [32129] WARNING: terminating connection
because of crash of another server process
2020-04-26 17:35:50.066 EEST [32129] DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.
2020-04-26 17:35:50.066 EEST [32129] HINT: In a moment you should be able
to reconnect to the database and repeat your command.
2020-04-26 17:35:50.067 EEST [32132] WARNING: terminating connection
because of crash of another server process
2020-04-26 17:35:50.067 EEST [32132] DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.
2020-04-26 17:35:50.067 EEST [32132] HINT: In a moment you should be able
to reconnect to the database and repeat your command.
2020-04-26 17:35:50.067 EEST [32142] WARNING: terminating connection
because of crash of another server process
2020-04-26 17:35:50.067 EEST [32142] DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.
2020-04-26 17:35:50.067 EEST [32142] HINT: In a moment you should be able
to reconnect to the database and repeat your command.
2020-04-26 17:35:50.068 EEST [32143] WARNING: terminating connection
because of crash of another server process
2020-04-26 17:35:50.068 EEST [32143] DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.
2020-04-26 17:35:50.068 EEST [32143] HINT: In a moment you should be able
to reconnect to the database and repeat your command.
2020-04-26 17:35:50.068 EEST [32140] WARNING: terminating connection
because of crash of another server process
2020-04-26 17:35:50.068 EEST [32140] DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.
2020-04-26 17:35:50.068 EEST [32140] HINT: In a moment you should be able
to reconnect to the database and repeat your command.
2020-04-26 17:35:50.069 EEST [32141] WARNING: terminating connection
because of crash of another server process
2020-04-26 17:35:50.069 EEST [32141] DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.
2020-04-26 17:35:50.069 EEST [32141] HINT: In a moment you should be able
to reconnect to the database and repeat your command.
2020-04-26 17:35:50.307 EEST [8385] LOG: all server processes terminated;
reinitializing
2020-04-26 17:35:52.355 EEST [32164] LOG: database system was interrupted;
last known up at 2020-04-26 17:30:29 EEST
2020-04-26 17:35:52.703 EEST [32164] LOG: database system was not properly
shut down; automatic recovery in progress
2020-04-26 17:35:52.705 EEST [32164] LOG: redo starts at 6FD/A53E0110
2020-04-26 17:35:53.885 EEST [32165] FATAL: the database system is in
recovery mode
2020-04-26 17:35:53.894 EEST [32166] FATAL: the database system is in
recovery mode
2020-04-26 17:35:55.285 EEST [32167] FATAL: the database system is in
recovery mode
2020-04-26 17:35:56.346 EEST [32168] FATAL: the database system is in
recovery mode
2020-04-26 17:35:57.416 EEST [32169] FATAL: the database system is in
recovery mode
2020-04-26 17:35:58.270 EEST [32164] LOG: redo done at 6FD/ED969B98
2020-04-26 17:35:58.466 EEST [32170] FATAL: the database system is in
recovery mode
2020-04-26 17:35:59.555 EEST [32171] FATAL: the database system is in
recovery mode
2020-04-26 17:36:00.606 EEST [32172] FATAL: the database system is in
recovery mode
2020-04-26 17:36:02.746 EEST [32174] FATAL: the database system is in
recovery mode
2020-04-26 17:36:03.409 EEST [8385] LOG: database system is ready to
accept connections
2020-04-26 17:36:03.805 EEST [32175] FATAL: the database system is in
recovery mode

Best regards,
Radu

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Magnus Hagander 2020-04-27 09:37:34 Re: PostgreSQL Server version compatibility check with Window 2016
Previous Message Paul Förster 2020-04-27 08:03:52 HOST variable in psql