Re: Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

På mandag 18. september 2017 kl. 16:28:07, skrev Bruce Momjian <bruce(at)momjian(dot)us
<mailto:bruce(at)momjian(dot)us>>:
On Sat, Sep 16, 2017 at 11:36:40PM +0200, Andreas Joseph Krogh wrote:
> På lørdag 16. september 2017 kl. 18:34:51, skrev Bruce Momjian <
> bruce(at)momjian(dot)us>:
>     No.  If you ran initdb with --waldir on the new primary, you will create
>     a symbolic link in the PGDATA directory, and a directory outside of
>     PGDATA for storing the WAL.  When you run rsync on the new primary
>     PGDATA directory, you will copy the symlink in the PGDATA directory, but
>     it will point to probably nothing on the standby.
>
>  
>  
> The misunderstanding here comes from the fact that I used pg_upgradecluster
> like this:
> pg_upgradecluster --method=upgrade --link 9.6 main
>  
> and it didn't issue initdb with --waldir on the new cluster (because
> pg_upgradecluster issues initdb for you), so pg_wal ended up in $PGDIR
because
> pg_upgradecluster didn't figure out the old cluster was initialized with
> --xlogdir. This is why I thought i made sense mentioning that one had to
move
> pg_wal manually.
>  
> I know it's debian-stuff, but maybe it's worth mentioning pg_upgradecluster
> somewhere and recommend not using it? It seems to start the new cluster
> automatically and when upgrading standbys section 10 tells you not to do
that.

So you didn't really follow the instructions, but instead are trying to
use the standby part of the instructions and found a problem with the
way pg_upgradecluster handled it.  We really can't document this.

It would be good to report the bug to pg_upgradecluster developers
though.

Yes, I can see rsync not working that case.
 
 
 
Actually I didn't know about --waldir switch of initdb and have always moved
pg_xlog manually then symlinking. 
 
 
 
>     > should be clearly pointed out that copying pg_wal is only needed in
those
>     > cases, and that it can be done with whatever network-copying procedure
>     you're
>     > familiar with, ie. scp/rsync. This step is not similar to the steps
>     required
>     > for copying tablespaces outside $PGDATA, so it's worth documenting
>     explicitly.
>     > Maybe also telling users to ensure the synlink (in $PGDATA) to pg_wal
on
>     > standby points to pg_wal.
>
>     Why tell them new instructions when the rsync instructions work fine?
>     What is the value?
>
>  
> The rsync instructions mentioned in 10.F all address the --link scenario and
> use "--delete --hard-links --size-only", and "merge 2 source-dirs into one",
> which isn't relevant when copying pg_wal.
>  
> This sentence:
> "If you have relocated pg_wal outside the data directories, rsync must be
run
> on those directories too."
> implies one must follow the rsync pattern elsewhere in 10.F, which isn't
really
> true. Maybe re-wording it to:
> "If you have relocated pg_wal outside the data directories you must copy it
> over to the new standby, and ensure the symlink from $PGDATA points to it"
> helps?

We can't document every possible configuration, especially if a
secondary tool is used in the middle.
 
 
But we're not talking about many different configurations, we're addressing
when pg_wal is located outside $PGDATA.
So it basically boils down to the last sentence in 10.F:
 
If you have relocated pg_wal outside the data directories, rsync must be run
on those directories too.
 
the word "must" here isn't correct. The point is that you have to copy the
waldir manually from the primary to the standby and ensure the symlink points
to this new location on the standby. So I still think something like this is
better: "If you have relocated pg_wal outside the data directories you must
copy it over to the new standby, and ensure the symlink from
$PGDATA/pg_wal points to it". I think it eliminates any doubt and makes the
instructions complete and easy to follow.
 
Thanks.

--
Andreas Joseph Krogh