Re: Shared WAL archive between master and standby: WALs not always identical

And also this:

"""
If archive_mode is set to on, the archiver is not enabled during recovery
or standby mode. If the standby server is promoted, it will start archiving
after the promotion, but will not archive any WAL it did not generate
itself. To get a complete series of WAL files in the archive, you must
ensure that all WAL is archived, before it reaches the standby
"""

If I understand it correctly, WAL started on master but finished on standby
(after failover) will still not be (old master is presumably dead and new
master must wait for next WAL segment). Of course, the next WAL segment
will be sent by new master, but we are going to miss exactly this one WAL
segment during which failover occurred and thus introduce interruption in
our WAL stream. Am I right?

Regards,
Sasa

On 28 February 2017 at 02:33, Sasa Vilic <sasavilic(at)gmail(dot)com> wrote:

> Hi David,
>
> thanks for the answer. I read this in documentation but here there is a
> corner case that I am not sure how to handle:
> """
> This requires more care in the archive_command, as it must be careful to
> not overwrite an existing file with different contents, *but return
> success if the exactly same file is archived twice.*
> """
> But what I am supposed to do when content differs? Still return success
> and ignore or return error? If I return error, wouldn't that prevent wal
> archiver slave from pushing further WALs?
>
> Regards,
> Sasa
>
>
> On 28 February 2017 at 02:10, David G. Johnston <
> david(dot)g(dot)johnston(at)gmail(dot)com> wrote:
>
>> On Mon, Feb 27, 2017 at 5:40 PM, Sasa Vilic <sasavilic(at)gmail(dot)com> wrote:
>>
>>> Aren't WALs from master and standby supposed to be identical?
>>>
>>
>> ​This would seem unwise to assume on its face and at least one piece of
>> documentation directly mentions that it is false:
>>
>> https://www.postgresql.org/docs/9.6/static/warm-standby.html
>> #CONTINUOUS-ARCHIVING-IN-STANDBY
>>
>> """
>> When continuous WAL archiving is used in a standby, there are two
>> different scenarios: the WAL archive can be shared between the primary and
>> the standby, or the standby can have its own WAL archive. When the standby
>> has its own WAL archive, set archive_mode to always, and the standby will
>> call the archive command for every WAL segment it receives, whether it's by
>> restoring from the archive or by streaming replication. *The shared
>> archive can be handled similarly, but the archive_command must test if the
>> file being archived exists already, and if the existing file has identical
>> contents*. This requires more care in the archive_command, as it must be
>> careful to not overwrite an existing file with different contents, but
>> return success if the exactly same file is archived twice. And all that
>> must be done free of race conditions, if two servers attempt to archive the
>> same file at the same time.
>> """
>>
>> ​The contents of both must match with respect to the data files but there
>> are likely things that go into the master WAL stream solely for the purpose
>> of communicating with a standby - ​and possibly some standby concepts that
>> would be unique to the standby's WAL - that would cause them to differ.
>> Not familiar enough to quickly list examples of what those might be. But
>> IIUC the system seems designed around master->slave replication and doesn't
>> support slave daisy-chains.
>>
>> David J.
>>
>>
>