Re: Possible missing segments in archiving on standby

On 2020/06/30 16:55, Kyotaro Horiguchi wrote:
> Hello.
>
> While looking a patch, I found that a standby with archive_mode=always
> fails to archive segments under certain conditions.

I encountered this issue, too.

> 1. v1-0001-Make-sure-standby-archives-all-segments.patch:
> Fix for A and B.
>
> 2. v1-0001-Make-sure-standby-archives-all-segments-immediate.patch:
> Fix for A, B and C.

You proposed two patches, but this patch should be reviewed preferentially
because this addresses all the issues (i.e., A, B and C) that you reported?

+ * If we are starting streaming at the beginning of a segment,
+ * there may be the case where the previous segment have not been
+ * archived yet. Make sure it is archived.

Could you clarify why the archive notification file of the previous
WAL segment needs to be checked?

As far as I read the code, the cause of the issue seems to be that
XLogWalRcvWrite() exits without creating an archive notification file
even if the current WAL segment is fully written up in the last cycle of
XLogWalRcvWrite()'s loop. That is, creation of the notification file
and WAL archiving of that completed segment will be delayed
until any data in the next segment is received and written (by next call
to XLogWalRcvWrite()). Furthermore, in that case, if walreceiver exits
before receiving such next segment, the completed current segment
fails to be archived as Horiguchi-san reported.

Therefore, IMO that the simple approach to fix the issue is to create
an archive notification file if possible at the end of XLogWalRcvWrite().
I implemented this idea. Patch attached.

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION