Re: pg_basebackup and pg_receivewal timing, missing WAL files

On 6/9/22 5:24 PM, Stephen Frost wrote:
>
> * Tim (timfosho(at)gmail(dot)com) wrote:
>> We currently use pgbackrest as our primary backup tool actually, and it
>> works great! The issue I ran into with trying to set it up that way with
>> pgbackrest, is when configuring the second repo, pgbackrest will have to
>> archive all WAL files to it, going back to the oldest backup. And we have
>
> Eh? Why is that? I think there's some confusion here regarding how
> pgbackrest multi-repo works. We won't archive back to the 'oldest
> backup' when you add a second repo, we'll just archive *new* WAL to it
> after it's been configured.

By default pgbackrest will keep all WAL since the start of the oldest
backup in a repo. Each repo can have its own retention settings so this
might be a *different* oldest WAL per repo.

However, you can use repo-retention-archive and
repo-retention-archive-type to change this behavior, see
https://pgbackrest.org/configuration.html#section-repository/option-repo-retention-archive.

Basically, if you need to keep a number of full backups for compliance
you can set repo-retention-archive-type=full and
repo-retention-archive-type=1 in that repo and WAL in between the
backups will only be kept after the last backup. WAL required to make
backups consistent is always kept until the backup expires.

We've had requests for repo-retention-archive-type=0, i.e. only archive
WAL during a backup, but we have not had time to implement that. Some
users have implemented a work-around based on checking the backup lock.
There is an issue that describes the method but I was not able to find
it after a brief search. This is safer than it sounds because pgbackrest
will check that all WAL required to make the backup consistent reached
the archive. So, if the wrapper is broken you'll get errors.

Regards,
--
-David
david(at)pgmasters(dot)net