Re: [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

Zeugswetter Andreas ADI SD wrote:
>
> > > The probably useful next step would be to pass the current length to
> the
> > > archive_command,
> > > so it can write the filled part of the file without the need for a
> > > filter.
> >
> > I can see that helping a lot, but not by writing onto the file on
> disk.
> > If the file is nearly empty, that would be a lot of disk I/O which
> doesn't
> > need to happen.
>
> I think you misunderstood what I meant.
> The actual archive command is constructed by expanding certain
> placeholders.
> I am suggesting to add such a placeholder for the size of the filled
> part of the log.
>
> A hypothetical example (note suggested %b placeholder for size in
> bytes):
> archive_command=dd if=%p of=/backup/WAL/%f bs=1 count=%b
>
> This allows to avoid unnecessary io for the backup of partially filled
> logs.

A nice improvement on that would be to have a "rearchive_command" to
allow to sync the new bytes written since a previous archive_command (so
it needs a new placeholder "start from this byte"). This allows writing
dd seek=%s skip=%s count=%b bs=1

(I had suggested something like this when PITR was just invented, but it
was disregarded because it was too complex for the first cut or the
feature).

--
Alvaro Herrera http://www.amazon.com/gp/registry/DXLWNGRJD34J
"Sallah, I said NO camels! That's FIVE camels; can't you count?"
(Indiana Jones)