Re: Streaming base backups

On 14.01.2011 13:38, Magnus Hagander wrote:
> On Fri, Jan 14, 2011 at 11:19, Heikki Linnakangas
> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>> On 14.01.2011 08:45, Fujii Masao wrote:
>>> 1. Smart shutdown is requested while walsender is sending a backup.
>>> 2. Shutdown causes archiver to end.
>>> (Though shutdown sends SIGUSR2 to walsender to exit, walsender
>>> running backup doesn't respond for now)
>>> 3. At the end of backup, walsender calls do_pg_stop_backup, which
>>> forces a switch to a new WAL file and waits until the last WAL file
>>> has
>>> been archived.
>>> *BUT*, since archiver has already been dead, walsender waits for
>>> that forever.
>>
>> Not only does it wait forever, but it writes the end-of-backup WAL record
>> after bgwriter has already exited and written the shutdown checkpoint
>> record.
>>
>> I think postmaster should treat a walsender as a regular backend, until it
>> has started streaming.
>>
>> We can achieve that by starting up the child as PM_CHILD_ACTIVE, and
>> changing the state to PM_CHILD_WALSENDER later, when streaming is started.
>> Looking at the postmaster.c, that should be safe, postmaster will treat a
>> backend as a regular backend anyway until it has connected to shared memory.
>> It is *not* safe to switch a walsender back to a regular process, but we
>> have no need to do that.
>
> Seems reasonable to me.
>
> I've applied a patch that exits base backups when the postmaster is
> shutting down - I'm happily waiting for Heikki to submit one that
> changes the shutdown logic in the postmaster :-)

Ok, committed a fix for that.

BTW, I just spotted a small race condition between creating a new table
space and base backup. We take a snapshot of all the tablespaces in
pg_tblspc before calling pg_start_backup(). If someone creates a new
tablespace and puts some data in it in the window between base backup
acquiring the list tablespaces and starting the backup, the new
tablespace won't be included in the backup.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com