BUG #18832: Segfault in GrantLockLocal

The following bug has been logged on the website:

Bug reference: 18832
Logged by: Robins Tharakan
Email address: tharakan(at)gmail(dot)com
PostgreSQL version: Unsupported/Unknown
Operating system: Ubuntu
Description:

While running some tests on a recent HEAD commit (d4a6c847ca), I got 3
segfaults today (2 for ALTER DEFAULT PRIVILEGES and 1 FOR SELECT
pg_drop_replication_slot()). I just happen to be testing this commit, so I
am not suspecting this commit to be the cause, but what I can add is that
I've run similar workloads last month and these crashes are too close to be
a coincidence - in effect I wouldn't be surprised if this is owing to a
recent change.

The crashes are frequent but not easy to reproduce (even with the exact same
database), so probably timing of concurrent queries is relevant, although
despite my best efforts I don't know / can't say what else runs concurrently
to cause this.

Thought I'd share here in case the backtraces helps someone triage this
faster.

SQL
===
The SQLs are entirely different, but they both crashed with almost exact
backtrace, thus clubbing them in this bug report.

ALTER DEFAULT PRIVILEGES FOR ROLE regress_dep_user1 IN SCHEMA deptest GRANT
ALL ON TABLES TO regress_dep_user2;

AND

SELECT pg_drop_replication_slot(...);

Commit
======
Not trivial to reproduce - so not much help here

Error Log - ALTER DEFAULT PRIVILEGES
=========
Occurence 1:
2025-03-06 05:19:43.945 ACDT [241648] LOG: client backend (PID 245735) was
terminated by signal 11: Segmentation fault
2025-03-06 05:19:43.945 ACDT [241648] DETAIL: Failed process was running:
ALTER DEFAULT PRIVILEGES FOR ROLE regress_dep_user1 IN SCHEMA deptest
GRANT ALL ON TABLES TO regress_dep_user2;
2025-03-06 05:19:43.945 ACDT [241648] LOG: terminating any other active
server processes
2025-03-06 05:19:43.957 ACDT [241648] LOG: all server processes terminated;
reinitializing
2025-03-06 05:19:44.126 ACDT [246034] FATAL: the database system is in
recovery mode
2025-03-06 05:19:44.126 ACDT [246033] LOG: database system was interrupted;
last known up at 2025-03-06 05:19:12 ACDT
2025-03-06 05:19:44.126 ACDT [246037] FATAL: the database system is in
recovery mode

Occurence 2:
2025-03-06 07:02:39.362 ACDT [441902] LOG: client backend (PID 446070) was
terminated by signal 11: Segmentation fault
2025-03-06 07:02:39.362 ACDT [441902] DETAIL: Failed process was running:
ALTER DEFAULT PRIVILEGES FOR ROLE regress_dep_user1 IN SCHEMA deptest
GRANT ALL ON TABLES TO regress_dep_user2;
2025-03-06 07:02:39.362 ACDT [441902] LOG: terminating any other active
server processes
WARNING: terminating connection because of crash of another server
process
DETAIL: The postmaster has commanded this server process to roll back the
current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and
repeat your command.
2025-03-06 07:02:39.367 ACDT [446319] FATAL: the database system is in
recovery mode

Backtrace - ALTER DEFAULT PRIVILEGES
=========
(Was able to capture backtrace only for the second crash)

Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `postgres: d4a6c847ca(at)sqith: smith postgres
127.0.0.1(48740) ALTER DEFAU'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00005ac747716802 in GrantLockLocal (locallock=0x5ac749e58398,
owner=0x5ac749df1718) at lock.c:1758
1758 lockOwners[i].owner = owner;
(gdb) bt
#0 0x00005ac747716802 in GrantLockLocal (locallock=0x5ac749e58398,
owner=0x5ac749df1718) at lock.c:1758
#1 0x00005ac747716a72 in GrantAwaitedLock () at lock.c:1840
#2 0x00005ac74772aa40 in LockErrorCleanup () at proc.c:814
#3 0x00005ac7472324f8 in AbortTransaction () at xact.c:2846
#4 0x00005ac7472330e2 in AbortCurrentTransactionInternal () at
xact.c:3495
#5 0x00005ac747233053 in AbortCurrentTransaction () at xact.c:3449
#6 0x00005ac74773fe0f in PostgresMain (dbname=0x5ac749df0e30 "postgres",
username=0x5ac749df0e18 "smith") at postgres.c:4408
#7 0x00005ac7477365ad in BackendMain (startup_data=0x7fffa06267f0,
startup_data_len=4) at backend_startup.c:107
#8 0x00005ac747640752 in postmaster_child_launch (child_type=B_BACKEND,
child_slot=70, startup_data=0x7fffa06267f0, startup_data_len=4,
client_sock=0x7fffa0626850)
at launch_backend.c:274
#9 0x00005ac74764721a in BackendStartup (client_sock=0x7fffa0626850) at
postmaster.c:3519
#10 0x00005ac7476446f3 in ServerLoop () at postmaster.c:1688
#11 0x00005ac747643fe9 in PostmasterMain (argc=3, argv=0x5ac749d6bc10) at
postmaster.c:1386
#12 0x00005ac7474e3db5 in main (argc=3, argv=0x5ac749d6bc10) at main.c:230
(gdb)

Backtrace Full - ALTER DEFAULT PRIVILEGES
==============
#0 0x00005ac747716802 in GrantLockLocal (locallock=0x5ac749e58398,
owner=0x5ac749df1718) at lock.c:1758
lockOwners = 0x0
i = 0
#1 0x00005ac747716a72 in GrantAwaitedLock () at lock.c:1840
No locals.
#2 0x00005ac74772aa40 in LockErrorCleanup () at proc.c:814
lockAwaited = 0x5ac749e58398
partitionLock = 0x797ddaff5480
timeouts = {{id = DEADLOCK_TIMEOUT, keep_indicator = false}, {id =
LOCK_TIMEOUT, keep_indicator = true}}
#3 0x00005ac7472324f8 in AbortTransaction () at xact.c:2846
s = 0x5ac747dba440 <TopTransactionStateData>
latestXid = 0
is_parallel_worker = false
__func__ = "AbortTransaction"
#4 0x00005ac7472330e2 in AbortCurrentTransactionInternal () at
xact.c:3495
s = 0x5ac747dba440 <TopTransactionStateData>
#5 0x00005ac747233053 in AbortCurrentTransaction () at xact.c:3449
No locals.
#6 0x00005ac74773fe0f in PostgresMain (dbname=0x5ac749df0e30 "postgres",
username=0x5ac749df0e18 "smith") at postgres.c:4408
local_sigjmp_buf = {{__jmpbuf = {140735884194072,
3455269497379230227, 3, 0, 99811950217592, 133581917298688,
3455269497461019155, 7314509273468768787}, __mask_was_saved = 1,
__saved_mask = {__val = {4194304, 99811983801536,
133581900226632, 4869, 99811983801536, 133581900226632, 643328, 0,
99811983801536, 4294967297, 99811983801536,
140735884191552, 99811943579575, 0, 99811983801536,
140735884191584}}}}
send_ready_for_query = false
idle_in_transaction_timeout_enabled = false
idle_session_timeout_enabled = false
__func__ = "PostgresMain"
#7 0x00005ac7477365ad in BackendMain (startup_data=0x7fffa06267f0,
startup_data_len=4) at backend_startup.c:107
bsdata = 0x7fffa06267f0
#8 0x00005ac747640752 in postmaster_child_launch (child_type=B_BACKEND,
child_slot=70, startup_data=0x7fffa06267f0, startup_data_len=4,
client_sock=0x7fffa0626850)
at launch_backend.c:274
pid = 0
#9 0x00005ac74764721a in BackendStartup (client_sock=0x7fffa0626850) at
postmaster.c:3519
bn = 0x797df60a1d38
pid = 1238813760
startup_data = {canAcceptConnections = CAC_OK}
cac = CAC_OK
__func__ = "BackendStartup"
#10 0x00005ac7476446f3 in ServerLoop () at postmaster.c:1688
s = {sock = 9, raddr = {addr = {ss_family = 2,
__ss_padding =
"\276d\177\000\000\001\000\000\000\000\000\000\000\000\230r\336I\307Z\000\000\001\000CM\000\000\000\000\230r\336I\307Z\000\000\020",
'\000' <repeats 33 times>, "CM\000\000\000\000\250v\336I\307Z", '\000'
<repeats 33 times>, __ss_align = 1296236544}, salen = 16}}
i = 0
now = 1741206717
last_lockfile_recheck_time = 1741206707
last_touch_time = 1741206707
events = {{pos = 1, events = 2, fd = 6, user_data = 0x0}, {pos = 0,
events = 0, fd = 6, user_data = 0x0}, {pos = 0, events = 0, fd = 1201168209,
user_data = 0x0}, {pos = 7,
events = 0, fd = 1024, user_data = 0x5ac749d73f50}, {pos =
1296236544, events = 0, fd = 0, user_data = 0x0}, {pos = 0, events = 0, fd =
0, user_data = 0x5ac749de7270}, {
pos = 0, events = 0, fd = 2048, user_data = 0x8}, {pos =
1238843216, events = 23239, fd = 1024, user_data = 0x438}, {pos =
1239306792, events = 23239, fd = 1239306808,
user_data = 0x4d430001}, {pos = 1239306792, events = 23239, fd =
16, user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x4d430002},
{pos = 1239306808,
events = 23239, fd = 8, user_data = 0x0}, {pos = 0, events = 0,
fd = 0, user_data = 0x4d430000}, {pos = 1239306808, events = 23239, fd = 8,
user_data = 0x0}, {pos = 0,
events = 0, fd = 1238844208, user_data = 0x7fffa0626b50}, {pos =
1201177455, events = 23239, fd = -1604162944, user_data = 0x5ac749d73f50},
{pos = 0, events = 0,
fd = 1024, user_data = 0x0}, {pos = 0, events = 0, fd =
1238843416, user_data = 0x0}, {pos = 5120, events = 0, fd = 1238844240,
user_data = 0x2d0}, {pos = 4, events = 0,
fd = 16, user_data = 0x5ac747984418 <wipe_mem+162>}, {pos = 784,
events = 0, fd = 1238843456, user_data = 0x0}, {pos = 0, events = 0, fd =
1296236545,
user_data = 0x5ac749d74040}, {pos = 784, events = 0, fd = 0,
user_data = 0x0}, {pos = 0, events = 0, fd = 1296236544, user_data =
0x5ac749d74040}, {pos = 784, events = 0,
fd = 0, user_data = 0x0}, {pos = 0, events = 0, fd = 3,
user_data = 0x81d41a496f152b00}, {pos = -1604162624, events = 32767, fd =
1201164451, user_data = 0x0}, {
pos = 1238843216, events = 23239, fd = 1296236544, user_data =
0xde7298}, {pos = 0, events = 0, fd = 1238843392, user_data =
0x5ac749d73f50}, {pos = 1024, events = 0,
fd = 88, user_data = 0x5ac749d74000}, {pos = 0, events = 0, fd =
1238843456, user_data = 0x7fffa0626c60}, {pos = 1201233573, events = 23239,
fd = 1203001705,
user_data = 0x5ac749d73f50}, {pos = 0, events = 0, fd = 0,
user_data = 0x1304}, {pos = 1238843216, events = 23239, fd = 0, user_data =
0x0}, {pos = 0, events = 0, fd = 0,
user_data = 0x1303}, {pos = 1238843216, events = 23239, fd = 0,
user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x7fffa0626c70},
{pos = 1863658240,
events = 2178161225, fd = -1604162384, user_data =
0x5ac749d89b20}, {pos = -1604162352, events = 32767, fd = 0, user_data =
0x5ac749de7270}, {pos = 0, events = 0, fd = 0,
user_data = 0x6be31}, {pos = -120, events = 4294967295, fd = 0,
user_data = 0x7fffa0626dc0}, {pos = -179355285, events = 31101, fd = 0,
user_data = 0x5ac749d89b20}, {
pos = 1203001688, events = 23239, fd = 0, user_data =
0x7fffa0626d20}, {pos = 1863658240, events = 2178161225, fd = 0, user_data =
0xffffffffffffff88}, {pos = 0,
events = 0, fd = 1239310928, user_data = 0x0}, {pos =
-155541504, events = 31101, fd = -1604162208, user_data = 0x797df54addae
<__GI___libc_free+126>}, {pos = 1203006005,
events = 23239, fd = 1239097824, user_data = 0x1da00000003},
{pos = -179739816, events = 31101, fd = 0, user_data = 0x81d41a496f152b00},
{pos = -179747632, events = 31101,
fd = 0, user_data = 0x0}, {pos = 0, events = 0, fd = 1238932456,
user_data = 0x81d41a496f152b00}, {pos = 1024, events = 0, fd = -1604161256,
user_data = 0x3}, {
pos = 1863658240, events = 2178161225, fd = -1604162112,
user_data = 0x797df544550d <__GI___sigprocmask+13>}, {pos = -1604161920,
events = 32767, fd = 1197737208,
user_data = 0x6be3147b46a35}, {pos = 0, events = 0, fd = 4868,
user_data = 0x5ac749d89b20}, {pos = 4194304, events = 0, fd = 0, user_data =
0x0}, {pos = 0, events = 0,

Error Log - SELECT pg_drop_replication_slot(...);
=========
2025-03-06 11:07:05.977 ACDT [740840] LOG: duration: 30222.816 ms
2025-03-06 11:07:06.596 ACDT [736676] LOG: client backend (PID 740584) was
terminated by signal 11: Segmentation fault
2025-03-06 11:07:06.596 ACDT [736676] DETAIL: Failed process was running:
SELECT pg_drop_replication_slot('regress_pg_walinspect_slot');
2025-03-06 11:07:06.596 ACDT [736676] LOG: terminating any other active
server processes
2025-03-06 11:07:06.606 ACDT [741206] FATAL: the database system is in
recovery mode

Backtrace - SELECT pg_drop_replication_slot(...);
=========
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00005840026dc7e4 in GrantLockLocal (locallock=0x5840169d7bc8,
owner=0x584016970b28) at lock.c:1758
1758 lockOwners[i].owner = owner;
(gdb) bt
#0 0x00005840026dc7e4 in GrantLockLocal (locallock=0x5840169d7bc8,
owner=0x584016970b28) at lock.c:1758
#1 0x00005840026dca54 in GrantAwaitedLock () at lock.c:1840
#2 0x00005840026f0a22 in LockErrorCleanup () at proc.c:814
#3 0x00005840021f84f8 in AbortTransaction () at xact.c:2846
#4 0x00005840021f90e2 in AbortCurrentTransactionInternal () at
xact.c:3495
#5 0x00005840021f9053 in AbortCurrentTransaction () at xact.c:3449
#6 0x0000584002705df1 in PostgresMain (dbname=0x584016970240 "postgres",
username=0x584016970228 "smith") at postgres.c:4408
#7 0x00005840026fc58f in BackendMain (startup_data=0x7ffc96df68c0,
startup_data_len=4) at backend_startup.c:107
#8 0x0000584002606752 in postmaster_child_launch (child_type=B_BACKEND,
child_slot=88, startup_data=0x7ffc96df68c0, startup_data_len=4,
client_sock=0x7ffc96df6920)
at launch_backend.c:274
#9 0x000058400260d21a in BackendStartup (client_sock=0x7ffc96df6920) at
postmaster.c:3519
#10 0x000058400260a6f3 in ServerLoop () at postmaster.c:1688
#11 0x0000584002609fe9 in PostmasterMain (argc=3, argv=0x5840168eac10) at
postmaster.c:1386
#12 0x00005840024a9db5 in main (argc=3, argv=0x5840168eac10) at main.c:230
(gdb)

Backtrace Full - SELECT pg_drop_replication_slot(...);
==============
#0 0x00005840026dc7e4 in GrantLockLocal (locallock=0x5840169d7bc8,
owner=0x584016970b28) at lock.c:1758
lockOwners = 0x0
i = 0
#1 0x00005840026dca54 in GrantAwaitedLock () at lock.c:1840
No locals.
#2 0x00005840026f0a22 in LockErrorCleanup () at proc.c:814
lockAwaited = 0x5840169d7bc8
partitionLock = 0x7272705f5600
timeouts = {{id = DEADLOCK_TIMEOUT, keep_indicator = false}, {id =
LOCK_TIMEOUT, keep_indicator = true}}
#3 0x00005840021f84f8 in AbortTransaction () at xact.c:2846
s = 0x584002d80440 <TopTransactionStateData>
latestXid = 0
is_parallel_worker = false
__func__ = "AbortTransaction"
#4 0x00005840021f90e2 in AbortCurrentTransactionInternal () at
xact.c:3495
s = 0x584002d80440 <TopTransactionStateData>
#5 0x00005840021f9053 in AbortCurrentTransaction () at xact.c:3449
No locals.
#6 0x0000584002705df1 in PostgresMain (dbname=0x584016970240 "postgres",
username=0x584016970228 "smith") at postgres.c:4408
local_sigjmp_buf = {{__jmpbuf = {140722839712232,
-2448216454042316899, 3, 0, 97031948511608, 125836300689408,
-2448216454115717219, -7962617830530183267},
__mask_was_saved = 1, __saved_mask = {__val = {4194304,
97032279600320, 125836284145736, 4869, 97032279600320, 125836284145736,
643328, 0, 97032279600320, 4294967297,
97032279600320, 140722839709712, 97031941873561, 0,
97032279600320, 140722839709744}}}}
send_ready_for_query = false
idle_in_transaction_timeout_enabled = false
idle_session_timeout_enabled = false
__func__ = "PostgresMain"
#7 0x00005840026fc58f in BackendMain (startup_data=0x7ffc96df68c0,
startup_data_len=4) at backend_startup.c:107
bsdata = 0x7ffc96df68c0
#8 0x0000584002606752 in postmaster_child_launch (child_type=B_BACKEND,
child_slot=88, startup_data=0x7ffc96df68c0, startup_data_len=4,
client_sock=0x7ffc96df6920)
at launch_backend.c:274
pid = 0
#9 0x000058400260d21a in BackendStartup (client_sock=0x7ffc96df6920) at
postmaster.c:3519
bn = 0x72728b550098
pid = 378453056
startup_data = {canAcceptConnections = CAC_OK}
cac = CAC_OK
__func__ = "BackendStartup"
#10 0x000058400260a6f3 in ServerLoop () at postmaster.c:1688
s = {sock = 9, raddr = {addr = {ss_family = 2,
__ss_padding = "\266
\177\000\000\001\000\000\000\000\000\000\000\000\030h\226\026(at)X\000\000\001\000CM\000\000\000\000\030h\226\026@X\000\000\020",
'\000'