Re: pg_basebackup + incremental base backups

On 21-May-20 08:43, Stephen Frost wrote:
> * Christopher Pereira (kripper(at)imatronix(dot)cl) wrote:
>
> [...]
>> Is there some way to rebuild the standby cluster by doing a differential
>> backup of the primary cluster directly?
> We've contemplated adding support for something like this to pgbackrest,
> since all the pieces are there, but there hasn't been a lot of demand
> for it and it kind of goes against the idea of having a proper backup
> solution, really.. It'd also create quite a bit of load on the primary
> to checksum all the files to do the comparison against what's on the
> replica that you're trying to update, so not something you'd probably
> want to do a lot more than necessary.

We have backups of the whole server and only need a efficient way to
rebuild the hot-standby cluster when pg_rewind is not able to do so.

I agree with your concerns about the increased load on the primary
server, but this rebuilding process would only be done in case of
emergency or during low load hours.

pg_basebackup works fine but does not support differential/incremental
backups which is a blocker.

Do you know any alternative software that is able to rebuild the standby
PG data dir using rsync or similar while the primary is still online?

It seems a simple pg_start_backup + rsync + pg_stop_backup (maybe
combined with a LVM snapshot) would do, but we would prefer to use some
existing tool.

We just tried barman, but it also seems to require a restore from the
backup before being able to start the standby server (?), and we are
afraid this would require double storage, IO and time for rebuilding the
standby cluster.

Thanks.