BUG #14326: Unexpected status after crash during exclusive backup

From: marco(dot)nenciarini(at)2ndquadrant(dot)it
To: pgsql-bugs(at)postgresql(dot)org
Subject: BUG #14326: Unexpected status after crash during exclusive backup
Date: 2016-09-16 09:54:48
Message-ID: 20160916095448.1421.65742@wrigleys.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

The following bug has been logged on the website:

Bug reference: 14326
Logged by: Marco Nenciarini
Email address: marco(dot)nenciarini(at)2ndquadrant(dot)it
PostgreSQL version: 9.6rc1
Operating system: Any
Description:

I was investigating a Postgres standby that was never reaching the
consistent recovery state and I discovered something unexpected in the
pg_controldata output:

Backup start location: 5E4/7C000028
Backup end location: 0/0

The standby was built using a cold backup of the master data directory, so I
was surprised to find "Backup start location" different from 0/0.

The replication was working correctly and the standby was perfectly aligned
with the master, moreover, the position 5E4/7C000028 was very old compared
to the latest checkpoint location, which was 675/78329748.

After further investigation I discovered that the cause of the issue was a
system crash which happened a month ago. Unfortunately when the system
crashed, an exclusive backup was running, so at restart it found a valid
backup_label and, given that the WAL file containing the backup start point
was still available, it started a backup recovery.

There is the issue: Postgres will never find the XLOG_BACKUP_END record
corresponding to the backupStartPoint recorded in control data, because it
was never written, so it will never reach the consistency point.

This has no user-visible effects unless the Postgres instance enters the
archive recovery state, in that case hot standby will never be activated.
Also, it doesn't impact any backup eventually taken from the instance
because to recover from the backup, the instance will go through a backup
recovery that will reset the backupStartPoint value.

The workaround I found to reset this state is to force the instance through
another backup recovery, by starting an exclusive backup, saving the
backup_label, stopping the backup and restarting the instance with the saved
backup_label in place.

I don't know the best way to handle this situation, but at least, I'd like a
warning message when the instance exits from the crash recovery while
backupStartPoint is still set.

This behaviour is present in every supported Postgres release and on master
as well.

Regards,
Marco

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message egor.pugin 2016-09-19 08:06:53 BUG #14327: UPSERT requires full path to column
Previous Message vbv256 2016-09-16 06:27:35 BUG #14325: backup restore inherited constraint