Re: PITR Recovery

On Wed, 2004-06-16 at 02:49, Tom Lane wrote:
> Simon Riggs <simon(at)2ndquadrant(dot)com> writes:
> > -finalaction refers to what to do when target is reached - the purpose
> > of this is to allow recovery of a database to occur when we don't have
> > enough space for all of the xlogs at once, so we need to do recovery in
> > batches.
>
> It seems to me that this is the only *essential* feature out of what
> you've listed, and the others are okay to add later. So I question
> your priorities:
>
> > In time for beta freeze, I think it is possible to do a limited subset
> > of the above:
> > - implement DATABASE only (whole instance, not specific database)
> > - implement END OF LOGS and TO TIMESTAMP
> > - implement THEN START only
> > - implement using simple C, rather than bison
>
> which seem to include everything except the one absolute must-have
> for any serious installation.
>

OK. At first, I disagreed, for many reasons.

I discussion with Bruce, I believe a fairly neat streaming solution is
possible.

During recovery, as each request for a new xlog is made, we can make a
system(3) call to a user defined recovery_program to retrieve the next
xlog and out it in place. As each xlog is closed the file will be
removed. The result of this would be to stream the data files through
recovery, so no more than 1-2 files would ever be required to perform
what could be (and is touted as this by other vendors) an infinite
recovery.

The result is that a backup tape (or other tape silo) could stream data
straight through to recovery, and would completely circumvent and
concern about insufficient disk space for recovery.

This would involve changes to XLogFileOpen() in xlog.c and far less
complex than I had imagined such functionality could be.

This could be specified to PostgreSQL by using:
- restore_program='cp %s %s' or similar

I'll work more on the design, but not tonight.

Best Regards, Simon Riggs