Re: Allowing multiple concurrent base backups

On Tue, Jan 11, 2011 at 22:51, Jeff Davis <pgsql(at)j-davis(dot)com> wrote:
> On Tue, 2011-01-11 at 22:56 +0200, Heikki Linnakangas wrote:
>> >   1. If it's a primary recovering from a crash, and there is a
>> > backup_label file, and the WAL referenced in the backup_label exists,
>> > then it does a bunch of extra work during recovery; and
>> >   2. In the same situation, if the WAL referenced in the backup_label
>> > does not exist, then it PANICs with a HINT to tell you to remove the
>> > backup_label.
>> >
>> > Is this an opportunity to solve these problems and simplify the code?
>>
>> It won't change the situation for pg_start_backup(), but with the patch
>> the base backups done via streaming won't have those issues, because
>> backup_label is not created (with that name) in the master.
>
> Do you think we should change the backup protocol for normal base
> backups to try to fix this? Or do you think that the simplicity of
> unrestricted file copy is worth these problems?
>
> We could probably make some fairly minor changes, like making a file on
> the primary and telling users to exclude it from any base backup. The
> danger, of course, is that they do copy it, and their backup is
> compromised.

I think keeping the flexibility is important. If it does add an extra
step I think that's ok once we have pg_basebackup, but it must be
reasonably *safe*. Corrupt backups from forgetting to exclude a file
seems not so.

But if the problem is you forgot to exclude it, can't you just remove
it at a later time?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/