Re: Backup solution over unreliable network

Greetings,

* Achilleas Mantzios (achill(at)matrix(dot)gatewaynet(dot)com) wrote:
> On 30/11/18 2:06 μ.μ., Stephen Frost wrote:
> >>- I have only read the best reviews about PgBackRest, can PgBackRest address those issues?
> >Glad to hear you've read good reviews about pgbackrest. As for
> >addressing these issues, pgbackrest has:
> >
> >- Backup resume
> >- Max WAL lag (in other words, you can have it simply start throwing WAL
> > away if it can't archive it, rather than allowing the primary to run
> > out of disk space)
>
> This is just superb! In our case we had the following architecture (now barman is defunct) :
>
> Primary (consistent snapshots with pg_start/stop_backup)+ --> reliable net (archive_command via rsync) --> WAL repository
>    | (async streaming replication)
>    | (reliable net)
>    V
> Standby --> unreliable net (barman via method rsync + barman streaming from standby ***) --> remote cloud provider site(barman)
>
> So Primary and Standby are in the same cloud provider over consistent
> (mostly) network, whereas the barman (remote recovery) site communicates
> over internet. We would like to keep the old functionality (or even add a
> new PgBackRest node in the main cloud provider, so the question is : is
> there a way for archive-push to two different stanzas? Or delegate the
> archive-push to work from the Standby ?

We've had a few folks using pgbackrest to push to two stanzas by way of
basically doing 'pgbackrest --stanza=a archive-push && pgbackrest
--stanza=b archive-push' and with that it does work, and you could
combine that with the max WAL setting, potentially, but it's not a
solution that I'm really a fan of. That's definitely a use-case we've
been thinking about though and have plans to support in the future,
but there are other things we're tackling now and so multi-repo hasn't
been a priority.

We've also considered supporting archive-mode=always and being able to
have the standby also push WAL and while we may support that in the
future, I'd say it's farther down on the list than multi-repo support.
As I recall, David Steele also had some specific technical concerns
around how to handle two systems pushing into the same WAL archive.
Having archive-mode=always be allowed if it's going to an independent
repo is an interesting thought though and might be simpler to do.

Thanks!

Stephen