Re: PITR logging control program

On Thu, Apr 29, 2004 at 07:34:47PM +0100, Simon Riggs wrote:

> Bruce is correct, the API waits for the archive to be full before
> archiving.
>
> I had thought about the case for partial archiving: basically, if you
> want to archive in smaller chunks, make your log files smaller...this is
> now a compile time option. Possibly there is an argument to make the
> xlog file size configurable, as a way of doing what you suggest.
>
> Taking multiple copies of the same file, yet trying to work out which
> one to apply sounds complex and error prone to me. It also increases the
> cost of the archival process and thus drains other resources.

My idea was basically that the archiver could be told "I've finished
writing XLog segment 1 until byte 9000", so the archiver would

dd if=xlog-1 seek=0 skip=0 bs=1c count=9000c of=archive-1

And later, it would get a notification "segment 1 until byte 18000" he does

dd if=xlog-1 seek=0 skip=0 bs=1c count=18000c of=archive-1

Or, if it's smart enough,

dd if=xlog-1 seek=9000c skip=9000c bs=1c count=9000c of=archive-1

Basically it is updating the logs as soon as it receives the
notifications. Writing 16 MB of xlogs could take some time.

When a full xlog segment has been written, a different kind of
notification can be issued. A dumb archiver could just ignore the
incremental ones and copy the files only upon receiving this other kind.

I think that if log files are too small, maybe it will be a waste of
resources (which ones?). Anyway, it's just an idea.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)