Re: wal-g (https://github.com/wal-g/wal-g) reliability

Greetings,

* Victor Sudakov (vas(at)sibptus(dot)ru) wrote:
> Nikolay Samokhvalov wrote:
> > In short, it's reliable and battle-tested. It's used in companies such as
> > Yandex.Cloud and GitLab.com, successfully.
>
> Nikolay, don't you mind another question?
>
> I'm used to the fact that "pg_basebackup -X" creates a self-sufficient
> backup of a cluster which can be started right away as it contains all
> the WAL files required for recovery. `touch recovery.signal` is never necessary,
> and `touch standby.signal` is optional (when you do PITR etc).
>
> It's not the case with wal-g, the result of the `wal-g backup-fetch`
> command requires `touch recovery.signal` and a restore_command
> configured to fetch WALs from the wal-g storage.
>
> I have also noticed that wal-g keeps pg_control in a separate tar
> archive, and keeps a lot of metadata.
>
> The questions are:
>
> 1. If the metadata in the wal-g storage ever becomes corrupt, will I be
> able to restore the database manually from the archives in
> $WALG_*_PREFIX/{basebackups,wal}_005/ ?
>
> 2. Is there a `wal-g backup-fetch` option for truly self-sufficient
> restoration?

Interesting that you ask this! I say that because it's actually a case
that pgbackrest contemplated and explicitly added support for-
specifically, if you set --archive-copy (or archive-copy=true) and
disable compression (or decompress everything before you start PG), then
you can just start PG from the backup and it'll perform the necessary
recovery from the WAL and start up. We considered this an interesting
use-case and used it extensively and have maintained support for it.

All that said, of course, this limits the ability to do typical PITR,
but at least it gives you a consistent backup which you can start PG
from.

Thanks,

Stephen