Re: Remove Deprecated Exclusive Backup Mode

Greetings,

* Christophe Pettus (xof(at)thebuild(dot)com) wrote:
> > On Feb 24, 2019, at 12:00, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> >
> > Do they realize how that existing backup strategy is flawed?
>
> Undoubtedly, some do, some don't. However, given that it has been the *only* backup API for a very long time, many organizations have spent a lot of time closing all of the holes.

No, it *hasn't* been the only backup API for a long time- that was only
true up until 9.6 was released, since then we've had both, and it's made
everything a downright mess because of exactly these arguments.

> It's not impossible to do safe backups with the existing API. Unquestionably, there are installations doing backups that might end up with a silently badly one, but it's entirely possible to do that unsafely (in many of the same ways) with the non-exclusively API.

Yes, it *is* impossible to do safe backups with the existing API. There
is an unquestionable race condition where a system restart will cause
your system to not come back up without you going in and removing the
backup_label file- and the only way you make that race window small is
to remove the backup_label file right after you run pg_start_backup and
copy it, and then PUT IT BACK at the end before you call pg_stop_backup,
which is insane, but otherwise the 'race window' is the ENTIRE length of
the backup.

This is the reason the non-exclusive backup mode *exists*. All of the
effort that went into building the non-exclusive backup mode wasn't done
just for the fun of it.

> The installations that need to fix the scripts are also exactly the ones that can't use pg_basebackup or another pre-packaged solution, usually because they have a specific way of taking the file system copy (SAN snapshot, etc.) that those don't support.

I'm not going to bother going into the issues that SAN snapshots have
here, because it's largely orthogonal to this discussion. What I'll say
is that they'll have over 5 years to adjust those scripts and I don't
see an issue with that.

> > We don't cater to this line of argument when it comes to breaking
> > changes in the backend, or when we break monitoring scripts, and I don't
> > see a reason why we should do so here.
>
> Those cases are not analogous.

I disagree *entirely*.

> 1. Backend APIs are declared non-stable, and do not have a wide audience compared to backing up the database.

*ALL* of our APIs are declared non-stable between major releases.
That's why we have *major* and *minor* releases.

> 2. Monitoring scripts, while important, are not as critical as the backup system. (And, in fact, I didn't agree with breaking those views either, but that's another discussion.)

Agreed, we should care even more about making sure that the tools we
provide to perform a backup are reliable, which is a good reason to
remove the broken and deprecated exclusive API.

> > Ok, then please do so, and please be prepared to continue to maintain
> > the documentation of both methods moving forward, because others have
> > tried and have (rightfully, in my opinion) decided that it's frankly not
> > worth the effort and ultimately just terribly confusing for users that
> > we have these two different backup methods and even just updating the
> > documentation for one or the other is downright painful (to the point
> > that people litterally give up on it).
>
> We're going to have to do that anyway. For as long as we are maintaining the documentation on a version that has both APIs, we're going to have to say, "Don't use this one, and *here's why*." Saying, "Don't use this one because we said so" when it is an API of long standing that works just as it always did isn't going to cut it.

We have existing documentation for the versions that have both APIs. It
sucks, it's horribly complicated, and no one wants to work on it because
of that.

Sure, it'd be nice if someone would fix it to list out the problems with
the exclusive API, but that requires someone wanting to spend time on
it. If that's not going to happen, I certainly don't think that means
we should *keep* it.

THanks!

Stephen