Re: Important lag in replication streaming between master and standby

On Mon, Aug 26, 2019 at 11:21 AM Mai Peng <maily(dot)peng(at)webedia-group(dot)com>
wrote:

> Hello,
>
> I’ve got several times this error : "requested WAL segment
> 000000020000A01A0000004F has already been remove". I think it’s due to the
> lag between our master in EU zone and our standby in US zone.
> How could we configure Postgres to handle this issue?
>

You can increase wal_keep_segments so that the master keeps more segments
around for the standy to stream. This has the risk that you still won't
set it high enough and so might not actually solve the problem.

You can use a replication slot, which will force the master to keep the
segments until the standby streams them. This has the risk that the pg_wal
can grow without bound (if the standby can't keep up, or if it goes
offline), and the master will fill up its storage and crash (running out of
storage on the pg_wal directory is not handled very gracefully).

You can set up a wal archive, and then set up the replica so it can fetch
from that archive as a backstop to streaming. As a bonus, WAL files
usually compress very well, and if you compress them during archival you
can save a lot of network bandwidth when the standby needs to retrieve
them.

Cheers,

Jeff