pg_rewind Issue: Trying to read Incorrect WAL file for checkpoint record

From: azeem subhani <azeemsd07(at)gmail(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: pg_rewind Issue: Trying to read Incorrect WAL file for checkpoint record
Date: 2024-07-13 11:33:11
Message-ID: CALp3DH1fLZmPvkOteAbUo4TOLZP-LstKOs6Gcw3Bm7acmJqk=w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi,

I have a 1 primary, 2 standby streaming replication cluster. The primary
node (`pgsql/16-b/data`) shut down to trigger a failover. One standby
(`psql/16-a/data`) was promoted to primary, and write operations were
performed on the new primary and replicated to the other standby.
I attempted to use pg_rewind to sync the failed primary with the current
primary, but it failed. The error shows pg_rewind is trying to read from an
incorrect WAL file `000000010000000000000015` instead of
`000000040000000000000015` as per the control data:

pg_rewind: servers diverged at WAL location 150000A0/0 on timeline 4
pg_rewind: error: could not open file
"/var/lib/pgsql/16-b/data/pg_wal/000000010000000000000015": No such file or
directory
pg_rewind: error: could not read WAL record at 0/15000028

*Failed Primary Control Data:*pg_control version number: 1300
Catalog version number: 202307071
Database system identifier: 7389684959150618064
Database cluster state: shut down
pg_control last modified: Sat 13 Jul 2024 08:50:32 AM +04
Latest checkpoint location: 0/15000028
Latest checkpoint's REDO location: 0/15000028
Latest checkpoint's REDO WAL file: 000000040000000000000015
Latest checkpoint's TimeLineID: 4
Latest checkpoint's PrevTimeLineID: 4

*Current Primary Control Data:*
pg_control version number: 1300
Catalog version number: 202307071
Database system identifier: 7389684959150618064
Database cluster state: in production
pg_control last modified: Sat 13 Jul 2024 12:09:43 PM +04
Latest checkpoint location: 0/15009590
Latest checkpoint's REDO location: 0/15009558
Latest checkpoint's REDO WAL file: 000000050000000000000015
Latest checkpoint's TimeLineID: 5
Latest checkpoint's PrevTimeLineID: 5

Could anyone provide guidance on resolving this issue? I cannot find the
WAL file that pg_rewind is trying to open and read in the data/pg_wal
directory or my WAL archive directory.
--
Thanks
Azeem Subhani

Browse pgsql-general by date

  From Date Subject
Next Message Peter J. Holzer 2024-07-13 22:05:45 Re: Dropping column from big table
Previous Message Muhammad Imtiaz 2024-07-13 09:34:21 Re: Replication lag in Postgres