Interaction of PITR backups and Bulk operations avoiding WAL

From: "Simon Riggs" <simon(at)2ndquadrant(dot)com>
To: <pgsql-hackers(at)postgresql(dot)org>
Subject: Interaction of PITR backups and Bulk operations avoiding WAL
Date: 2007-03-09 13:55:48
Message-ID: 1173448548.3641.276.camel@silverbirch.site
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Reviewing earlier threads, I realised that there was a potential
bug/loophole in PITR backups in conjunction with avoiding WAL for bulk
operations. This would be rare, but should be fixed.
http://archives.postgresql.org/pgsql-hackers/2006-05/msg01113.php

Say you issue COPY, CREATE INDEX etc..
pg_start_backup()
pg_stop_backup()
...then bulk operation ends.
This will result in a base backup that does not contain the data written
during the bulk operation and the changes aren't in WAL either.

I propose to fix this by making two new calls
bool RequestBulkCommandUseNoWAL(void)
void ResetBulkCommandUseNoWAL(void)

so we would use it like this
use_wal = RequestBulkCommandUseNoWAL()
and then at end of operation
if (!use_wal)
ResetBulkCommandUseNoWAL();

The routine would record a flag on the shmem ControlFile data that would
prevent pg_start backup functions from executing while a bulk operation
was in progress. It would also prevent a bulk operation from using no
WAL while a backup was in progress, as is already the case, since the
backup can only take place while archiving is enabled.

A new entry point pg_start_backup(text, bool) would allow the user to
specify whether to wait for bulk ops to finish, or not. The old entry
point would always wait, to ensure safety in all cases.

Thoughts?

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Greg Smith 2007-03-09 13:57:48 Re: Log levels for checkpoint/bgwriter monitoring
Previous Message Simon Riggs 2007-03-09 13:54:58 Re: Auto creation of Partitions