Re: Problem with PITR recovery

Simon Riggs wrote:
> On Mon, 2005-04-18 at 21:25 -0400, Bruce Momjian wrote:
> > Tom Lane wrote:
> > > Simon Riggs <simon(at)2ndquadrant(dot)com> writes:
> > > > The wal file could be truncated after the log switch record, though I'd
> > > > want to make sure that didn't cause other problems.
> > >
> > > Which it would: that would break WAL file recycling.
> >
> > Good point. I don't see non-full WAL archiving as a problem for the
> > backup or shutdown, but I do see an issue with doing archives every X
> > seconds. If someone sets that really low (and someone will) we could
> > easily fill the disk.
>
> The disk would only fill if the archiver doesn't keep up with
> transmitting xlog files to the archive. The archive can fill up if it is
> not correctly sized, even now. Switching log files every N seconds would
> at least give a very predictable archive sizing calculation which should
> actually work against users sizing their archives poorly.

I was thinking of the archiver filling because of lots of almost-empty
16mb files. If you archive every five seconds, it is 11 Gigs/hour,
which is not too bad, I guess, but I would bet compression would save
space and I/O load too.

> > However, rather than do it ourselves, maybe we
> > should make it visible to administrators so they know exactly what is
> > happening and can undo it in case they need to recover, something like:
> >
> >
> > archive_command = 'gzip <%p >%f'
> >
> > so the compression is done in a way that is visible to the
> > administrator.
>
> As long as we tell them there's more than one way to do it. Many tape
> drives offer hardware compression, for example, so there would be no
> gain in doing this twice.

Good point. I am thinking 'gzip --fast' would be the best option for
copies to another file system. I see about 0.6 seconds to compress a
16mb WAL file here and I get 16x compression.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073