Re: Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

On Tue, Jun 9, 2015 at 1:04 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> On Tue, Jun 9, 2015 at 9:09 AM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>>
>> On Tue, May 12, 2015 at 10:42 PM, Andrew Dunstan <andrew(at)dunslane(dot)net>
>> wrote:
>> > Map basebackup tablespaces using a tablespace_map file
>> >
>> > Windows can't reliably restore symbolic links from a tar format, so
>> > instead during backup start we create a tablespace_map file, which is
>> > used by the restoring postgres to create the correct links in pg_tblspc.
>> > The backup protocol also now has an option to request this file to be
>> > included in the backup stream, and this is used by pg_basebackup when
>> > operating in tar mode.
>> >
>> > This is done on all platforms, not just Windows.
>> >
>> > This means that pg_basebackup will not not work in tar mode against 9.4
>> > and older servers, as this protocol option isn't implemented there.
>>
>> While I was performing the recovery-related tests, I found that there was
>> the case where $PGDATA/tablespace_map was not renamed to *.old at the
>> recovery.
>> Is this harmless and intentional?
>
> There shouldn't be any problem because we tablespace_map file
> only if backup file is present.
>
>> Sorry if this has been already
>> discussed so far.
>>
>> The steps to reproduce the problem is:
>>
>> 1. start base backup, i.e., call pg_start_backup().
>> 2. repeat some write transactions and checkpoints until the WAL file
>> containing
>> the checkpoint record that backup_label indicates will be removed.
>> 3. killall -9 postgres
>> 4. start the server and a crash recovery.
>>
>> At this time, the crash recovery fails with the following error messages.
>> 2015-06-09 12:26:54 JST FATAL: could not locate required checkpoint
>> record
>> 2015-06-09 12:26:54 JST HINT: If you are not restoring from a backup,
>> try removing the file ".../backup_label".
>>
>> 5. according to the above hint message, remove $PGDATA/backup_label
>> and restart a crash recovery
>>
>> Then you can see that tablespace_map remains in $PGDATA after the recovery
>> ends.
>>
>
> The basic idea is that tablespace_map file will be used in case we
> have to restore from a backup which contains tablespaces. So
> I think if user is manually removing backup_label, there is no
> meaning of tablespace_map, so that should also be removed. One
> way to address this is modify the Hint message suggesting that
> remove tablespace_map if present.
>
> Current :
> If you are not restoring from a backup, try removing the file
> \"%s/backup_label\
> Modify it to:
> If you are not restoring from a backup, try removing the file
> \"%s/backup_label\
> and, if present \"%s/tablespace_map\.

Or what about removing tablespace_map file at the beginning of recovery
whenever backup_label doesn't exist? I'd like to avoid making a user do
such manual operation if possible.

Regards,

--
Fujii Masao