Re: pg_basebackup and pg_receivewal timing, missing WAL files

>
> However, you can use repo-retention-archive and
> repo-retention-archive-type to change this behavior, see
>
> https://pgbackrest.org/configuration.html#section-repository/option-repo-retention-archive
> .
> Basically, if you need to keep a number of full backups for compliance
> you can set repo-retention-archive-type=full and
> repo-retention-archive-type=1 in that repo and WAL in between the
> backups will only be kept after the last backup. WAL required to make
> backups consistent is always kept until the backup expires.

So as I understand it, it would keep WALs from the most current backup up
to the most current WALs.
In my case, the only purpose of this 2nd repo is to keep monthly backups
that are 2-6 months old (and occasionally test restores)
for legal purposes.
The most recent backup in this repo would still be 2 months old, so it
would be keeping an unnecessary 2 months of WALs.

We've had requests for repo-retention-archive-type=0, i.e. only archive
> WAL during a backup, but we have not had time to implement that. Some
> users have implemented a work-around based on checking the backup lock.
> There is an issue that describes the method but I was not able to find
> it after a brief search. This is safer than it sounds because pgbackrest
> will check that all WAL required to make the backup consistent reached
> the archive. So, if the wrapper is broken you'll get errors.

I'm guessing you're referring to: optional wal archiving · Issue #900 ·
pgbackrest/pgbackrest (github.com)
<https://github.com/pgbackrest/pgbackrest/issues/900>,
This seems a bit tricky to get right (and will take some work to convince
my team that its legit), also doesn't look immediately possible to only
omit WAL archiving on one repo out of multiple.

Definitely a +1 here for implementing that feature
(repo-retention-archive=0). pgbackrest's ability to stream into Azure
storage & simultaneously compress
using parallel CPUs is absolutely killer. It's also limiting us from using
it in our non production databases, as
we run those in Docker containers, and don't want any WAL archiving. Being
able to quickly configure and store 1 off backups
for an often changing environment would be great.

Stephen Frost wrote:

> You just need to handle the modification of the pgbackrest config to
> add/remove the second repo (and run the backup to the second repo when
> it's enabled) when you want a new backup of it.

I will have to try this method of dynamically tweaking the configuration,
did not consider that.

Thank you for all your help everyone!

On Thu, Jun 9, 2022 at 6:37 PM David Steele <david(at)pgmasters(dot)net> wrote:

> Ugh, typos.
>
> On 6/9/22 6:34 PM, David Steele wrote:
> > On 6/9/22 5:24 PM, Stephen Frost wrote:
> >
> > Basically, if you need to keep a number of full backups for compliance
> > you can set repo-retention-archive-type=full and
> > repo-retention-archive-type=1 in that repo and WAL in between the
>
> Here I meant repo-retention-archive-type=full and repo-retention-archive=1.
>
> >
> > We've had requests for repo-retention-archive-type=0, i.e. only archive
>
> And here repo-retention-archive=0.
>
> Regards,
> --
> -David
> david(at)pgmasters(dot)net
>