Re: PITR Backup state validity checking

Simon Riggs <simon(at)2ndquadrant(dot)com> writes:
> Solution Design:
> Before a backup is taken, write a file to data directory that identifies
> which backup this is. When the backup is taken this file will be copied
> with the backup, and later restored when the backup is restored.
> When backup completes write a file to xlog directory that contains the
> start backup identifier and the end time.

End WAL offset, please. Let's not waste our time with imprecise thinking.
(If you want to throw in the time too, as an aid to the DBA, fine, but
the correctness check wants the WAL position.)

> - Backup may be taken when postmaster is down, so solution must not rely
> on postmaster being up.

Sure it can. If postmaster is not up then the whole problem is
immaterial, as the WAL state isn't changing relative to the database
state. (It might be a good idea to try to have some kind of interlock
that prevents someone from trying to start the postmaster while such
a backup is in progress.)

I'm not convinced that we need expend a whole lot of effort on the
point, though, as surely people will prefer to keep their postmasters
running while they take backups.

> - It *is* posssible to do incremental backups, as long as the backup
> checks each file's change data against files already archived (or on
> write-once media).

Hmmm ... if you trust file change dates I suppose this would work,
but it feels shaky ...

> Implement an external program, called pg_backup.

I'd prefer to keep this inside the postmaster, as a separate program
offers a whole new set of failure modes (wrong version, wrong idea about
where PGDATA is, etc etc).

> - Alter archiver to always archive backup_end* files first, so they are
> written to archive in time sequence order.

We cannot use an archive technique that does not support requests for
arbitrary files, so your concern about write ordering seems quite
pointless. These backup ID files will have roles similar to timeline ID
files, which already require random access.

> I'd certainly prefer a solution that involved writing WAL records to
> indicate start and end, which seems cleaner and more integrated.

But harder to use. The DBA would find it much more convenient to have
those items of info out in easily readable text files.

regards, tom lane