Proposal to add page headers to SLRU pages

From: "Li, Yong" <yoli(at)ebay(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: "Shyrabokau, Anton" <antons(at)ebay(dot)com>, "Debnath, Shawn" <sdn(at)ebay(dot)com>
Subject: Proposal to add page headers to SLRU pages
Date: 2023-12-07 07:06:50
Message-ID: IA1PR07MB982925731C4C0B38DC8F2F7FB98BA@IA1PR07MB9829.namprd07.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi all,

PostgreSQL currently maintains several data structures in the SLRU cache. The current SLRU pages do not have any header, so it is impossible to checksum a page and verify its integrity. It is very difficult to debug issues caused by corrupted SLRU pages. Also, without a page header, page LSN is tracked in an ad-hoc fashion using LSN groups, which requires additional data structure in the shared memory. At eBay, we are building on the patch shared by Rishu Bagga in [1], which adds the standard PageHeaderData to each SLRU page. We believe that adding the standard page header to each SLRU page is the correct approach for the long run. It adds a checksum to each SLRU page, tracks page LSN as if it is a standard page and eases future page enhancements.

The enclosed patch changes the address calculation logic for all 7 SLRUs in the following 6 files:
src/backend/access/transam/clog.c
src/backend/access/transam/commit_ts.c
src/backend/access/transam/multixact.c
src/backend/access/transam/subtrans.c
src/backend/commands/async.c
src/backend/storage/lmgr/predicate.c

The patch enables page checksum with changes to the following 2 files:
src/backend/access/transam/slru.c
src/bin/pg_checksums/pg_checksums.c

The patch removes the group LSNs defined for each SLRU cache. See changes to:
src/include/access/slru.h

The patch adds a few helper macros in the following files:
src/backend/storage/page/bufpage.c
src/include/storage/bufpage.h

The patch updates some test cases:
src/bin/pg_resetwal/t/001_basic.pl
src/test/modules/test_slru/test_slru.c

I am still working on patching the pg_upgrade. Just love to hear your thoughts on the idea and the current patch.

Discussed with: Anton Shyrabokau and Shawn Debnath

[1] https://www.postgresql.org/message-id/flat/EFAAC0BE-27E9-4186-B925-79B7C696D5AC%40amazon.com

Regards,
Yong

Attachment Content-Type Size
slru_add_page_header.patch application/octet-stream 43.4 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2023-12-07 07:07:42 Re: Remove MSVC scripts from the tree
Previous Message David Rowley 2023-12-07 06:33:13 Re: catalog access with reset GUCs during parallel worker startup