BUG #18669: pg_get_shmem_allocations show wrong memory for 11 structures

From: PG Bug reporting form <noreply(at)postgresql(dot)org>
To: pgsql-bugs(at)lists(dot)postgresql(dot)org
Cc: o15611(at)gmail(dot)com
Subject: BUG #18669: pg_get_shmem_allocations show wrong memory for 11 structures
Date: 2024-10-23 15:15:07
Message-ID: 18669-2c19adfa8eb52bfc@postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

The following bug has been logged on the website:

Bug reference: 18669
Logged by: Oleg Ivanov
Email address: o15611(at)gmail(dot)com
PostgreSQL version: 17.0
Operating system: Linux
Description:

pg_shmem_allocations hide real sizes for 11 structures, mainly hash
tables.

select * from (select *, lead(off) over(order by off) - off as diff from
pg_shmem_allocations) as a where a.diff<>a.allocated_size order by 5 desc;

name | off | size |
allocated_size | diff
------------------------------------------------+-----------+------+----------------+---------
PREDICATELOCK hash | 144359936 | 2896 |
2944 | 1260416
Shared Buffer Lookup Table | 141591296 | 2896 |
2944 | 928640
LOCK hash | 142520064 | 2896 |
2944 | 695168
PROCLOCK hash | 143215232 | 2896 |
2944 | 695168
PREDICATELOCKTARGET hash | 143914624 | 2896 |
2944 | 445312
RWConflictPool | 145883904 | 24 |
128 | 292992
PredXactList | 145620352 | 88 |
128 | 205184
Proc Header | 146444672 | 136 |
256 | 114944
SERIALIZABLEXID hash | 145825536 | 2896 |
2944 | 58368
WaitEventCustom hash by wait event information | 147903488 | 2896 |
2944 | 6400
WaitEventCustom hash by name | 147909888 | 2896 |
2944 | 6400
(11 rows)

diff is the real size. "Shared Buffer Lookup Table" (Buffer Mapping Hash)
can not be 2944 byte for 128

Robert Haas shown similar idea to check the correct sizes, but used wrong
query, so hiddnig the real sizes was not obviouse.
https://www.postgresql.org/message-id/CA%2BTgmoazhJnk25jKPXtRPxNddM8ZxDdJ7-OP5wiWSBsXJ2AWBg%40mail.gmail.com

select *, off - lag(off + allocated_size, 1) over () as hole_size from
(select * from pg_shmem_allocations order by 2) x order by 5 desc limit
14;
name | off | size | allocated_size
| hole_size
-------------------------------------+-----------+---------+----------------+-----------
DSM Registry Data | 54656 | 16 | 128
|
<anonymous> | | 4739584 | 4739584
|
PredXactList | 145620352 | 88 | 128
| 1257472
Buffer Strategy Status | 142519936 | 28 | 128
| 925696
Fast Path Strong Relation Lock Data | 143910400 | 4100 | 4224
| 692224
PROCLOCK hash | 143215232 | 2896 | 2944
| 692224
PREDICATELOCK hash | 144359936 | 2896 | 2944
| 442368
FinishedSerializableTransactions | 146176896 | 16 | 128
| 292864
SERIALIZABLEXID hash | 145825536 | 2896 | 2944
| 205056
Proc Array | 146559616 | 524 | 640
| 114688
RWConflictPool | 145883904 | 24 | 128
| 55424
WaitEventCustom hash by name | 147909888 | 2896 | 2944
| 3456
| 147916288 | 1939968 | 1939968
| 3456
Buffer Descriptors | 5731072 | 1048576 | 1048576
| 0
(14 rows)

For checking:

select * from pg_shmem_allocations order by off;
name | off | size |
allocated_size
------------------------------------------------+-----------+-----------+----------------
...
Checkpoint BufferIds | 141263616 | 327680 |
327680
Shared Buffer Lookup Table | 141591296 | 2896 |
2944
Buffer Strategy Status | 142519936 | 28 |
128
...

142519936-141591296=928640 - lie: not 2944
141591296-141263616=327680 - true.

Hidding memory can be a security hole wich may be used to inject a delayed
multi-step vulnerability.
I suggest to calculate and show the real sizes.

Browse pgsql-bugs by date

  From Date Subject
Next Message Alexandra Wang 2024-10-23 16:19:14 Re: pg_rewind fails on Windows where tablespaces are used
Previous Message jian he 2024-10-23 14:18:22 Re: BUG #18568: BUG: Result wrong when do group by on partition table!