Re: Remove Deprecated Exclusive Backup Mode

On Wed, Jul 1, 2020 at 10:28 PM David Steele <david(at)pgmasters(dot)net> wrote:

> On 7/1/20 3:58 PM, Robert Haas wrote:
> > On Wed, Jul 1, 2020 at 3:50 PM Magnus Hagander <magnus(at)hagander(dot)net>
> wrote:
> >> As far as I've seen, the one thing that people have problems with in
> the exclusive mode backups are precisely the fact that they have to keep a
> persistent conneciton open, and thus it cannot work together with backup
> software that is limited to only supporting running a pre- and a post
> script.
> >>
> >> Something like I have suggested here is to solve *that* problem. I
> don't think anybody actually explicitly wants "exclusive backups" -- they
> want a backup solution that plugs into their world of pre/post scripts. And
> if we can make that one work in a safer way than the current exclusive
> backups, ohw is that not an improvement?
> >
> > Yeah, I guess that's a pretty fair point. I have to confess to having
> > somewhat limited enthusiasm for adding a third mode here, but it might
> > be worth it.
> >
> > It seems pretty well inevitable to me that people are going to forget
> > to end them. I am not sure exactly what the consequences of that will
> > be, but if for example there's a limited number of shared memory slots
> > to store information about these backups, then if you leak any, you'll
> > eventually run out of slots and your backups will start failing. I
> > feel like that's a going to happen to about 75% of the people who try
> > to use this new backup mode at some point in time, but maybe I'm a
> > pessimist.[1]
>
> I did consider going down this road and I came to more or less the same
> conclusion. Backups would accumulate until we just ran out of space
> and/or started throwing errors at some limit. Hardly very attractive.
>
> Also, full pages writes will be left on indefinitely.
>
> > If we could jigger things so that you don't need to stop the backup at
> > all, you only start it, and whether you ever finish copying everything
> > is something about which the system need not know or care, that would
> > be a lot nicer. I'm not sure I see how to do that, though.
>
> Well, the only thing pg_stop_backup() *really* needs to know is the
> starting WAL position. pg_start_backup() gets that info so if it passes
> it back to pg_stop_backup() that could be enough. Or as was proposed
> above, it just passes the backup_label back to pg_stop_backup() for
> parsing. To write the .backup file to WAL (which I personally think is
> pretty useless) you'd need backup_label.
>
> The issue I have not been able to work around is that full page writes
> need to be on for backups and they won't get turned back off if you
> don't end all the backups that get started.
>
> Here's a thought. What if we just stored the oldest starting LSN and a
> count of how many backups have been requested. When the backup ends it
> checks that backup count is > 0 and starting LSN is <= its starting LSN.
> If not, it throws an error. When backups go to 0 FPWs are turned off if
> they were off before the first backup.
>

I guess the weak spot of that one is if some script does stop without doing
start first, it will break somebody else's backup. (And yes, I've seen
scripts make this mistake many times -- it equally breaks the exclusive
backups in the current system...)

And don't we need the combination of the start/stop location for the
history file?

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>