RE: BUG #17326: Postgres crashed when pg_reload_conf() with ssl certificate parameters

try to install debug_info and get stack,
1. use coredump ,
]$ gdb -q -c /pgdata/core.1317550.sig11.1639122870s /usr/pgsql-13/bin/postgres
Reading symbols from /usr/pgsql-13/bin/postgres...Reading symbols from .gnu_debugdata for /usr/pgsql-13/bin/postgres...(no debugging symbols found)...done.
(no debugging symbols found)...done.

warning: Can't open file (null) during file-backed mapping note processing

warning: Can't open file (null) during file-backed mapping note processing

warning: Can't open file (null) during file-backed mapping note processing
[New LWP 1317550]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/pgsql-13/bin/postgres'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f72e3290094 in asn1_string_embed_free () from /lib64/libcrypto.so.1.1

2. when gdb log ,
Program received signal SIGHUP, Hangup.
0x00007f4fb438e25b in select () from /lib64/libc.so.6
Continuing.

Program received signal SIGHUP, Hangup.
0x00007f4fb438e25b in select () from /lib64/libc.so.6
Continuing.

Program received signal SIGSEGV, Segmentation fault.
0x00007f4fb5eef094 in asn1_string_embed_free () from /lib64/libcrypto.so.1.1
Continuing.

Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.

Should I install debug info for set_user module too?

Thanks,

James

-----Original Message-----
From: James Pang (chaolpan)
Sent: Thursday, December 9, 2021 11:34 AM
To: Dmitry Dolgov <9erthalion6(at)gmail(dot)com>
Cc: pgsql-bugs(at)lists(dot)postgresql(dot)org
Subject: RE: BUG #17326: Postgres crashed when pg_reload_conf() with ssl certificate parameters

Looks like this issue is related with "set_user" extension, I removed all extensions , pg_reload_conf() works without issue. When I installed and enable "set_user" extension, the issue got reproduced.
shared_preload_libraries = 'orafce,pgaudit,pg_cron,pg_stat_statements,pg_prewarm,set_user'
#set_user
set_user.superuser_whitelist = '+dba'
#set_user.superuser_allowlist = '+dba'
set_user.block_log_statement=on
set_user.nosuperuser_target_whitelist = ''
#set_user.nosuperuser_target_allowlist = ''

Will try to get and update the stack.

James

-----Original Message-----
From: Dmitry Dolgov <9erthalion6(at)gmail(dot)com>
Sent: Wednesday, December 8, 2021 9:46 PM
To: James Pang (chaolpan) <chaolpan(at)cisco(dot)com>
Cc: pgsql-bugs(at)lists(dot)postgresql(dot)org
Subject: Re: BUG #17326: Postgres crashed when pg_reload_conf() with ssl certificate parameters

> The following bug has been logged on the website:
>
> Bug reference: 17326
> Logged by: James Pang
> Email address: chaolpan(at)cisco(dot)com
> PostgreSQL version: 13.4
> Operating system: RHEL8.4
> Description:
>
> we need SSL enabled for our production env, when I test renew a ssl
> certificate , and reload_conf, it crashed. even with same certificate
> and ssl parameters, run reload_conf often lead to Postgres crash. For
> example
> :
>
> =# select name,setting from pg_settings where name like 'ssl_%' order by name;
> name | setting
> ----------------------------------------+-----------------------------
> ----------------------------------------+--
> ----------------------------------------+--------
> ssl_ca_file |
> /var/lib/pgsql/sslcerts/awstestca.crt
> ssl_cert_file |
> /var/lib/pgsql/sslcerts/server.crt
> ssl_ciphers | HIGH:MEDIUM:+3DES:!aNULL
> ssl_crl_file |
> ssl_dh_params_file |
> ssl_ecdh_curve | prime256v1
> ssl_key_file |
> /var/lib/pgsql/sslcerts/server.key
> ssl_library | OpenSSL
> ssl_max_protocol_version |
> ssl_min_protocol_version | TLSv1.2
> ssl_passphrase_command |
> ssl_passphrase_command_supports_reload | off
> ssl_prefer_server_ciphers | on
> (13 rows)
>
> =# select pg_reload_conf();
> pg_reload_conf
> ----------------
> t
> (1 row)
>
> =# select pg_reload_conf();
> pg_reload_conf
> ----------------
> t
> (1 row)
>
> =# select pg_reload_conf();
> FATAL: terminating connection due to unexpected postmaster exit server closed the connection unexpectedly
> This probably means the server terminated abnormally
> before or while processing the request.
> The connection to the server was lost. Attempting reset: Failed.
>
> On Wed, Dec 08, 2021 at 06:22:11AM +0000, James Pang (chaolpan) wrote:
> From postgres logs , it show
> 2021-12-08 03:57:55.826 UTC::@:[1291058]:[9-1]:2021-12-08 03:33:21
> UTC:LOG: received SIGHUP, reloading configuration files
> 2021-12-08 03:58:02.832 UTC::@:[1291058]:[10-1]:2021-12-08 03:33:21
> UTC:LOG: received SIGHUP, reloading configuration files
> 2021-12-08 03:58:03.143
> UTC:10.240.212.242(58646):jamet(at)jamet:[1291076]:[9-1]:2021-12-08
> 03:33:24 UTC:testsubLOG: disconnection: session time: 0:24:38.967
> user=jamet database=jamet host=10.240.212.242 port=58646
> 2021-12-08 03:58:03.147
> UTC:[local]:postgres(at)jamet:[1291397]:[3-1]:2021-12-08 03:57:02
> UTC:psqlFATAL: terminating connection due to unexpected postmaster
> exit
> 2021-12-08 03:58:03.147
> UTC:[local]:postgres(at)jamet:[1291397]:[4-1]:2021-12-08 03:57:02
> UTC:psqlLOG: disconnection: session time: 0:01:00.405 user=postgres
> database=jamet host=[local]

Hi,

Thanks for reporting the issue. Any chance to get a stack trace corresponding to the crash, e.g. like in [1]?

[1]: https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD