Re: Streaming replication and WAL archive interactions

On 04/22/2015 12:42 AM, Robert Haas wrote:
> On Tue, Apr 21, 2015 at 6:55 AM, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
>> On 04/21/2015 12:04 PM, Michael Paquier wrote:
>>> On Tue, Apr 21, 2015 at 4:38 PM, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
>>> wrote:
>>>> Note that even though we don't archive the partial last segment on the
>>>> previous timeline, the same WAL is copied to the first segment on the new
>>>> timeline. So the WAL isn't lost.
>>>
>>> But if the failed master has archived those segments safely, we may need
>>> them, no? I am not sure we can ignore a user who would want to do a PITR
>>> with recovery_target_timeline pointing to the one of the failed master.
>>
>> I think it would be acceptable. If you want to maintain an up-to-the-second
>> archive, you can use pg_receivexlog. Mind you, if the standby wasn't
>> promoted, the partial segment would not be present in the archive anyway.
>> And you can copy the WAL segment manually from 0000000200000000000000XX to
>> pg_xlog/0000000100000000000000XX before starting PITR.
>>
>> Another thought is that we could archive the partial file, but with a
>> different name to avoid confusing it with the full segment. For example, we
>> could archive a partial 000000010000000000000012 segment as
>> "000000020000000000000012.00000128.partial", where 00000128 indicates how
>> far that file is valid (this naming is similar to how the backup history
>> files are named). Recovery wouldn't automatically pick up those files, but
>> the DBA could easily copy the partial file into pg_xlog with the full
>> segment's name, if he wants to do PITR to that piece of WAL.
>
> So, suppose you A replicating to B (via an archive) replicating to C
> (via a separate archive); A dies, B is promoted. It sounds to me like
> today this will work and with your proposed change it will require
> manual intervention.

No. If there is no streaming replication involved, no partial files will
be archived, with or without this patch. There is no change to that
scenario.

Note that it's a bit complicated to set up that scenario today.
Archiving is never enabled in recovery mode, so you'll need to use a
custom cron job or something to maintain the archive that C uses. The
files will not automatically flow from B to the second archive. With the
patch we're discussing, however, it would be easy: just set
archive_mode='always' in B.

- Heikki