Re: Updated backup APIs for non-exclusive backups

On Wed, Feb 10, 2016 at 4:38 PM, David Steele <david(at)pgmasters(dot)net> wrote:

> On 2/10/16 7:46 AM, Magnus Hagander wrote:
> > Per discussion at the developer meeting in Brussels, here's a patch that
> > makes some updates to the backup APIs, to support non-exclusive backups
> > without using pg_basebackup. <...>
>
> This sounds like a great idea and I have signed up to review.
>
> > * A new version of pg_stop_backup is created, taking an argument
> > specifying if it's exclusive or not. The current version of
> > pg_stop_backup() continues to work for exclusive backups, with no
> > changes to behavior. The new pg_stop_backup will return a record of
> > three columns instead of just the value -- the LSN (pglsn), the backup
> > label file (text) and the tablespace map file (text). If used to cancel
> > an exclusive backup, backup label file and tablespace map will be NULL.
> > At this point it's up to the backup software to write down the files in
> > the location of the backup.
>
> It would be nice if this new pg_stop_backup() function also output the
> exact time when the backup stopped. Depending on how long
> pg_stop_backup() waits for WAL to be archived a time-stamp recorded
> after pg_stop_backup() returns can be pretty far off.
>
> This information is handy for automating selection of a backup when
> doing time-based PITR (or so the user can manually select). For
> exclusive backups it is possible to parse the .backup file to get this
> information but non-exclusive backups do not output the .backup file.
>

The non-exclusive backups *do* output the backup_label file, it just shows
up as a text field instead of as a separate file. You're supposed to write
that data to a file in the backup program.

> I would be happy to see the time-stamp returned from the
> pg_start_backup() function as well. It's a bigger change, but once
> pg_start_backup() returns multiple columns it will be easier to add more
> columns in the future.
>
> In fact, it might be better if backup_label and tablespace_map were
> output by pg_start_backup(). This would give the backup software more
> information to work with from the start.
>

I was trying to figure out why that's a bad idea, but I can't really say
why :)

For pg_basebackup backups, I think the reason we put it at the end is
simply that we don't want to write it into the backup directory until the
rest of the backup is complete, since it's not going to be useful before
then. But that requirement can certainly be put on the backup client
instead of the server. With the newer API it's going to have to keep those
things around anyway.

That would then change the function syntax for pg_start_backup() but
instead make pg_stop_backup() now behave the same as it did before.

Would anybody else like to comment on if that's a good idea or if there are
any holes in it? :)

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/