Re: Remove Deprecated Exclusive Backup Mode

On Thu, Jul 2, 2020 at 10:35 AM Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>
wrote:

> On Wed, 2020-07-01 at 16:46 -0400, Stephen Frost wrote:
> > I continue to find it curious that we stress a great deal over (very
> > likely) poorly written backup scripts that haven't been updated in the
> > 5+? years since exclusive mode was deprecated, but we happily add new
> > keywords in new major versions and remove columns in our catalog tables
> > or rename columns or entirely redo how recovery works (breaking every
> > script out there that performed a restore..) or do any number of other
> > things that potentially break applications that communicate through the
> > PG protocol and other parts of the system that very likely have scripts
> > that were written to work with them.
> >
> > I'm really of the opinion that we need to stop doing that.
> >
> > If someone could explain what is so special about *this* part of the
> > system that we absolutely can't possibly accept any change that might
> > break user's scripts, and why it's worth all of the angst, maintenance,
> > ridiculously difficult documentation to understand and hacks (the
> > interface to pg_start_backup is ridiculously warty because of this), I'd
> > greatly appreciate it.
>
> Easy. Lots of people write backup scripts, and fewer people write
> complicated pg_catalog queries. We would make way more users unhappy.
>

It's not about complicated catalog queries.

For example, in PostgreSQL 13 we break what I would say is the vast
majority of every query against pg_stat_statements, by renaming those
fields. Yet the old ones were never deprecated, and certainly not for 5+
years. And I certainly *hope* we have more users who query
pg_stat_statements than write their own backup solutions.

In PostgreSQL 12, we broke every single restore script by moving backup
configuration into recovery.conf. Yet the old ones were never deprecated,
and certainly not for 5+ years.

Or for PostgreSQL 10, we renamed pg_xlog to pg_wal, and pg_log to log, both
which would break the vast majority of existing backup scripts. Yet the old
ones were never deprecated, and certainly not for 5+ years.

In general, we have made backwards incompatible changes in almost every
release of PostgreSQL. And the vast majority of them have *not* been
deprecated for years ahead of making the change -- we just killed them.

So I do agree with Stephen on this -- it seems we are holding this old and
if not broken then at least very *fragile* API to a *very* different set of
standards than our other functionality.

All backup scripts had to be rewritten in 10. All restore scripts had to be
rewritten in 12. What's so special about requiring backup scripts to be
rewritten again in 14?

Now, I understand it if we were removing functionality that you could not
rewrite to make things work. Heck, I can even at least partially understand
that non-exclusive backup mode is too *difficult* for some people to use.
But the currently suggested changes are there specifically to make it as
easy to use as the deprecated one, yet a lot safer.

Besides, there is nothing "poorly written" about a backup script using
> exclusive backup as long as the user is aware that there could be some
> small degree of trouble in case of a crash.
>

It's not a poorly written script, but it *is* a poorly designed API. But if
we decide we can never fix things that we have designed poorly, then what
other parts of our development should we stop?

(And no, I don't agree it's a "small degree of trouble". At all. Because
usually you end up with a total system outage sometime in the middle of the
night (when most people run their backups), blocking end user access,
waking up on-call personnel, and dealing with a pretty damn major incident)

But one instance where we made users unhappy is not justification
> for making them unhappy again.
>

This is true.

So which users will actually be unhappy about this?

Yes, they have to make small updates to their scripts. Again, if they don't
want that, they can *never* upgrade PostgreSQL, because we make such
changes in *every* major release. Those are small, and those are trivial
(with the suggested changes, not today). In return, they get a database
that won't crash on them if something happens during backup.

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>