Re: Incremental backup from a streaming replication standby fails

From: Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: David Steele <david(at)pgmasters(dot)net>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Incremental backup from a streaming replication standby fails
Date: 2024-07-24 10:46:52
Message-ID: e3f9b4be04881d353e3ad8206776d4e9707026e4.camel@cybertec.at
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, 2024-07-22 at 09:37 -0400, Robert Haas wrote:
> On Fri, Jul 19, 2024 at 6:07 PM Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at> wrote:
> > Here is a patch.
> > I went for both the errhint and some documentation.
>
> Hmm, the hint doesn't end up using the word "standby" anywhere. That
> seems like it might not be optimal?

I guessed that the user was aware that she is taking the backup on
a standby server...

Anyway, I reworded the hint to

This can happen for incremental backups on a standby if there was
little activity since the previous backup.

> Hmm. I feel like I'm about to be super-nitpicky, but this seems
> imprecise to me in multiple ways.

On the contrary, cour comments and explanations are valuable.

> How about something like this:
>
> An incremental backup is only possible if replay would begin from a
> later checkpoint than for the previous backup upon which it depends.
> On the primary, this condition is always satisfied, because each
> backup triggers a new checkpoint. On a standby, replay begins from the
> most recent restartpoint. As a result, an incremental backup may fail
> on a standby if there has been very little activity since the previous
> backup. Attempting to take an incremental backup that is lagging
> behind the primary (or some other standby) using a prior backup taken
> at a later WAL position may fail for the same reason.
>
> I'm not saying that's perfect, but let me know your thoughts.

I tinkered with this some more, and the attached patch has

An incremental backup is only possible if replay would begin from a later
checkpoint than the checkpoint that started the previous backup upon which
it depends. If you take the incremental backup on the primary, this
condition is always satisfied, because each backup triggers a new
checkpoint. On a standby, replay begins from the most recent restartpoint.
Therefore, an incremental backup of a standby server can fail if there has
been very little activity since the previous backup, since no new
restartpoint might have been created.

Yours,
Laurenz Albe

Attachment Content-Type Size
v2-0001-Add-documentation-and-hint-for-incremental-backup.patch text/x-patch 2.8 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Heikki Linnakangas 2024-07-24 10:55:02 Re: Speed up JSON escape processing with SIMD plus other optimisations
Previous Message torikoshia 2024-07-24 10:43:37 Re: Add on_error and log_verbosity options to file_fdw