Re: pg_receivewal - couple of improvements

On Mon, Feb 7, 2022 at 8:23 AM Michael Paquier <michael(at)paquier(dot)xyz> wrote:
>
> On Sun, Feb 06, 2022 at 01:01:41PM +0530, Bharath Rupireddy wrote:
> > With the start position as an option, users can just provide the LSN
> > from which they want to stream the WAL, in the above case, it can be
> > from primary's latest checkpoint LSN.
>
> This still strikes me as a dangerous thing to have, prone to errors
> with a bunch of base backups wasted if one does a mistake as it would
> be very easy to cause holes in the WAL stored, until one has to
> redeploy from a base backup in urgency. This kind of control is
> provided by replication slots for new locations, and the current
> archive location if anything is stored, so I would leave it at that.

What if someone doesn't use pg_receivewal as an archive location? The
pg_receivewal can also be used for synchronous replication quorum
right? In this situation, I don't mind if some of the WAL files are
missing in pg_receivewal's target directory, but I don't want to do
the manual work of getting the WAL files to the pg_receivewal's target
directory from my archive location to make the pg_receivewal up and
connect with primary again? The primary can still remove the WAL files
needed by pg_receivewal (after the max_slot_wal_keep_size limit). If I
can tell pg_receivewal where to get the start position, then that will
be a good idea.

> On top of that, this kind of control is prone to more race conditions
> with the backend, as a concurrent checkpoint could make the LSN you
> are looking for irrelevant.

I understand that having a start position as an option is more
error-prone and creates holes in the WAL file. Why can't we allow
users to choose the current start position calculation of the
pg_receivewal? Users can choose to tell pg_receivewal either to get
start position from its target directory or from its slot's
restart_lsn or from the server's identify_system command. Thoughts?

Regards,
Bharath Rupireddy.