Re: pg_receivewal - couple of improvements

On Thu, Feb 03, 2022 at 06:40:55PM +0530, Bharath Rupireddy wrote:
>
> Isn't complex for anyone to go to the archive location which involves
> extra steps - getting authentication tokens, searching there for the
> required WAL file, downloading it, unzipping it, copying back to
> pg_receivewal node etc. in production environments? You know, this
> will just be problematic and adds more time for bringing up the
> pg_receivewal. Instead if I know that the latest checkpoint LSN and
> archived WAL file from the primary, I can just provide the startpos
> (probably the last checkpoint LSN) to pg_receivewal so that it can
> continue getting the WAL records from primary, avoiding the whole
> bunch of the manual work that I had to do.

I don't get it. If you're missing WAL it means that will you have to do that
tedious manual work to retrieve them no matter what. So on top of that tedious
work, you also have to make sure that you don't provide a bogus start position.

Also, doesn't this scenario implies that you have both archive_command and
pg_receivewal for storing your WALs elsewhere? In my experience this isn't
really common.

If you want to make sure you won't have to do that tedious work of retrieving
the WALs from a different location, you should probably rely on the facilities
to make sure it won't happen, like using a replication slots and monitoring the
pg_wal usage.

Maybe that's a good idea but I'm still having a hard time imagining a scenario
where it would actually be a good idea.