Re: recovery_target_time and WAL fetch with streaming replication

Sorry to revive this post, but I have the same problem:

I set a streaming replication slave with this steps:
1) create a physical replication slot in master
2) pg_basebackup -S slot (...)
3) create a recovery.conf with primary_conninfo, primary_slot_name and
recovery_min_apply_delay = '4d'
4) start replica and wal_receiver is started and write the wal files
5) after 4 days, I have a delayed replica and ALL wal files to apply a
recovery from 4 days until now

The problem is: if I restart the master or the replica, the
wal_receiver will not start and the replica will not receive
the new WAL files because all files in the pg_wal match the
recovery_min_apply_delay criteria.
Master will retain the WAL files (because the inactive slot and last
LSN), growing the space to save this wals.

But the real problem is if the master die: I have a replica with only
WAL files until restart.

To solve, I need to receive the WAL by pg_receivewal and use a
traditional cp in recovery.conf to apply them.
But this is an extra work to monitor and cleanup this WALs.

There is a way to start walreceiver independent from
recovery_min_apply_delay set ?

Best regards,

Alexandre

Em dom, 13 de mai de 2018 às 08:45, Michael Paquier
<michael(at)paquier(dot)xyz> escreveu:
>
> On Sun, May 13, 2018 at 09:42:42AM +0200, Hannes Erven wrote:
> > But when new WAL is needed, the standby will fetch /all/ WAL present on the
> > master.
>
> Fetching as much WAL as possible when recovery happens is wanted by
> design, so as it recovers as much as possible. And that's documented.
>
> > I'd say, the standby should either:
> > - always connect to the primary and fetch any WAL present
>
> This is what a hot standby does. It keeps waiting for WAL to become
> available whichever the source used (archive, local pg_xlog or stream)
> and switches between one or the other. You can look at
> WaitForWALToBecomeAvailable to get an idea of the details.
>
> > - stop fetching/streaming WAL when it is not needed for the current
> > recovery_target
>
> The startup process is in charge of recovery (WAL replay and definition
> of from where to get the WAL available), and is the one which decides if
> using streaming is necessary or not. if streaming is used, then it
> starts a WAL receiver. If a switch from streaming to another WAL source
> (local pg_xlog or archives is done), then it shuts down the WAL
> receiver, consisting in sending SIGTERM to the WAL receiver and stopping
> it immediately with a FATAL message (stops process immediately). The
> key point is that WAL receiver is designed to be a light-weight
> transport layer for WAL data. In short, to be simple, it receives a set
> of WAL bytes and writes them. It does not include any logic to decode
> WAL records, so it cannot know when a stop point happens or not. It
> also has no idea of the configuration within recovery.conf, which is
> loaded by the startup process.
>
> > Yes, but thats far less simple than just setting restore_target_time .
>
> It seems to me that archiving provides the control you are looking for.
> --
> Michael