Re: recovery_target_time and WAL fetch with streaming replication

On Sun, May 13, 2018 at 09:42:42AM +0200, Hannes Erven wrote:
> But when new WAL is needed, the standby will fetch /all/ WAL present on the
> master.

Fetching as much WAL as possible when recovery happens is wanted by
design, so as it recovers as much as possible. And that's documented.

> I'd say, the standby should either:
> - always connect to the primary and fetch any WAL present

This is what a hot standby does. It keeps waiting for WAL to become
available whichever the source used (archive, local pg_xlog or stream)
and switches between one or the other. You can look at
WaitForWALToBecomeAvailable to get an idea of the details.

> - stop fetching/streaming WAL when it is not needed for the current
> recovery_target

The startup process is in charge of recovery (WAL replay and definition
of from where to get the WAL available), and is the one which decides if
using streaming is necessary or not. if streaming is used, then it
starts a WAL receiver. If a switch from streaming to another WAL source
(local pg_xlog or archives is done), then it shuts down the WAL
receiver, consisting in sending SIGTERM to the WAL receiver and stopping
it immediately with a FATAL message (stops process immediately). The
key point is that WAL receiver is designed to be a light-weight
transport layer for WAL data. In short, to be simple, it receives a set
of WAL bytes and writes them. It does not include any logic to decode
WAL records, so it cannot know when a stop point happens or not. It
also has no idea of the configuration within recovery.conf, which is
loaded by the startup process.

> Yes, but thats far less simple than just setting restore_target_time .

It seems to me that archiving provides the control you are looking for.
--
Michael