Re: pitr replica dies on startup

On Fri, 2007-08-31 at 21:56 -0400, Tom Lane wrote:
> Jeff Frost <jeff(at)frostconsultingllc(dot)com> writes:
> > That all seems reasonable enough. Is it in the docs somewhere? I
> > didn't find anything like this mentioned. If not, could we get it
> > added as a note?
>
> Yeah, it hadn't occurred to anyone to specify this, because we just
> thought of recovery_command as fetching from a static archive.
> We clearly need to document the expected semantics better.
>
> I'm wondering whether we should discourage people from putting
> side-effects into the recovery_command, period. You already found out
> that recovery can ask for the same file more than once, but what if it
> never asks for a particular file at all? I'm not sure that can happen,
> just playing devil's advocate.

Yes, side effects are bad, in general.

> Simon, did you see this thread? What do you think?

The two main things we want are:
1. avoid doing a COPY from the archive, to improve performance
2. automatically clear down the archive when running in continuous
recovery mode

At first thought, it seems easy to fix this by removing the double file
request. That idea was in my original PITR patch, but it was removed and
sensibly so. The subtle point is that if you remove files from the
archive too early then you may prevent the recovery from being
restarted. So "mv" should never be used to avoid the copy operation,
because it implements (1) but not (2).

I've submitted a patch that sends the recovery_command a new %r value
which is the last file needed to restart the recovery. All files prior
to that file can be removed from the archive. That is then used by
pg_standby to maintain the archive. We discussed the patch as being for
8.3, but the patch is currently in the 8.4 queue. That patch solves this
issue, so I'd ask that we review that now. (pgsql-patches, 8 April).
That includes initial docs on pg_standby, which does implement file
management correctly.

There *is* a bug in PITR which I fixed recently. (pgsql-patches, 8
June). That one really needs to be in 8.3 - more review work, sorry.

Whatever we do, there's more docs coming on this for 8.3, pg_standby,
pg_compresslog and a few other points. Give me a week or so please.

--
Simon Riggs
2ndQuadrant http://www.2ndQuadrant.com