Re: pg_basebackup from cascading standby after timeline switch

On Fri, Dec 21, 2012 at 9:54 PM, Heikki Linnakangas
<hlinnakangas(at)vmware(dot)com> wrote:
> Yes, this should be backpatched to 9.2. I came up with the attached.

In this patch, if '-X stream' is specified in pg_basebackup, the timeline
history files are not backed up. We should change pg_backup background
process and walsender so that they stream also timeline history files,
for example, by using 'TIMELINE_HISTORY' replication command?
Or basebackup.c should send all timeline history files at the end of backup
even if '-X stream' is specified?

> However, thinking about this some more, there's a another bug in the way WAL
> files are included in the backup, when a timeline switch happens.
> basebackup.c includes all the WAL files on ThisTimeLineID, but when the
> backup is taken from a standby, the standby might've followed a timeline
> switch. So it's possible that some of the WAL files should come from
> timeline 1, while others should come from timeline 2. This leads to an error
> like "requested WAL segment 00000001000000000000000C has already been
> removed" in pg_basebackup.
>
> Attached is a script to reproduce that bug, if someone wants to play with
> it. It's a bit sensitive to timing, and needs tweaking the paths at the top.
>
> One solution to that would be to pay more attention to the timelines to
> include WAL from. basebackup.c could read the timeline history file, to see
> exactly where the timeline switches happened, and then construct the
> filename of each WAL segment using the correct timeline id. Another approach
> would be to do readdir() on pg_xlog, and include all WAL files, regardless
> of timeline IDs, that fall in the right XLogRecPtr range. The latter seems
> easier to backpatch.

The latter sounds good to me. But how all WAL files with different timelines
are shipped in pg_basebackup -X stream mode?

Regards,

--
Fujii Masao