Re: Setting up continuous archiving

From: Benoit Lobréau <benoit(dot)lobreau(at)gmail(dot)com>
To: yuri(dot)kanivetsky(at)gmail(dot)com
Cc: ptim007(at)yahoo(dot)com, pgsql-general(at)lists(dot)postgresql(dot)org
Subject: Re: Setting up continuous archiving
Date: 2018-10-15 16:09:55
Message-ID: CAPE8EZ4hVaYJeRGz58XJz2X2TF=y7X1bL9dSUQLJBn1wdeScFg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

By the way, do/can they both use streaming to receive WAL records? Or
> streaming is only for standby servers. For backups you have only
> file-based log shipping?
>

barman supports streaming but it's not as magical as one might think.
See pgbarman's documentation for how to manager .partial files.

pgbackrest archives only wal files when postgres uses the archive_command.
You also have the option to do paralllel async wal push/get.
It can be useful if you write wals quicker than you can archive them or if
you want to restore more quickly.

Then, I suppose they both don't support partial PITR
> (http://docs.pgbarman.org/release/2.4/#scope) where there are
> standalone backups that extends to points in time for which there are
> no WAL files. I'm not sure if this matters, but I assume that it might
> be effective in terms of disk space.
>

It's a hot backup so you have to have wals files so that your backup is
consistent at the end of the backup.
You can build something like what you describe with pgbackrest,
archive-copy and a copy of the backup directory to another place.

> Like, base backups + WAL files covering the last month, and a couple
> of standalone backups for a couple of months before that. Compared to
> base backups + WAL files covering the same period of time.
>

I see. Yes keeping the wal for months can take some space...

By that you mean, why use repmgr, that targets specifically PostgreSQL
> in place of Pacemaker + Corosync which are more general pieces of
> software?
>
I'd like to be able to handle db failure as fast as possible. Ideally,
> automatically. Which probably means either repmgr, or corosync +
> pacemaker + PAF. Is that what you mean by HA here?

yes. You can also look into patrony for an alternative. It's a matter of
preference and requirements.

Set up like corosync & pacemaker can be a little complex at first. But HA
is not a trivial matter. And fencing is tricky to set up correctly.
If you require Hight availability it means you value your data and what
your service up. So it's a non sense to go quick and cheap. You should try
each and see for your self. (I contributed to PAF so I am not unbiased)

> Or at least, have a
> running instance I can switch to manually. Which means, for example,
> pgBackRest's streaming replication.
>

I am not following here. switchover doesn't requiert a backup tool. For
your switchover you just have to stop the master (so that all wal file are
sent to online stanbies) and promote the standby.

If your standby is lagging too far behind (and you dont use slots) you can
use pgbackrest's archive to fill the gap in wal files and catchup with the
master.

regards,
Benoit

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Adrian Klaver 2018-10-15 16:18:17 Re: Regarding varchar max length in postgres
Previous Message Phil Endecott 2018-10-15 15:57:41 Filtering before join with date_trunc()