Re: pg_receivewal - couple of improvements

On Sun, Feb 6, 2022 at 12:16 PM Michael Paquier <michael(at)paquier(dot)xyz> wrote:
>
> On Thu, Feb 03, 2022 at 10:01:42PM +0800, Julien Rouhaud wrote:
> > I don't get it. If you're missing WAL it means that will you have to do that
> > tedious manual work to retrieve them no matter what. So on top of that tedious
> > work, you also have to make sure that you don't provide a bogus start position.
>
> I may be wrong in saying that, but the primary use case I have seen
> for pg_receivewal is a service integration for archiving.

Not only for archiving, but it can also be used as a
synchronous/asynchronous standby.

> > Maybe that's a good idea but I'm still having a hard time imagining a scenario
> > where it would actually be a good idea.
>
> With the defaults that we have now in place (current LSN location,
> current slot's location or the archive location), I am not really
> convinced that we need more control in this area with the proposed
> option.

Having the start position as an option for pg_receivewal can be useful
in scenarios where the LSN/WAL file calculated from the pg_receivwal's
target directory is removed by the primary (for whatever reasons). In
such scenarios, we have to manually remove the WAL files(a risky thing
IMO) in the pg_receivwal's target directory so that the pg_receivewal
will calculate the next start position from the slot info and then
from the server position via IDENTIFY_SYSTEM command.

With the start position as an option, users can just provide the LSN
from which they want to stream the WAL, in the above case, it can be
from primary's latest checkpoint LSN.

I still feel that the start position as an option would be a good
addition here, so that users can choose the start position.

If others don't agree with the start position as an option, at least
having the current start position calculation (first, from
pg_receivewal's target directory, if not, from slot's restart_lsn, if
not, from server's identifiy_system command) as a user specified
option will be of great help. Users can say, 'calculate start position
from target directory or slot's restart_lsn or server's
identify_system command).

Regards,
Bharath Rupireddy.