Re: File based Incremental backup v8

Hi Robert,

2015-03-07 2:57 GMT+11:00 Robert Haas <robertmhaas(at)gmail(dot)com>:

> By the way, unless I'm missing something, this patch only seems to
> include the code to construct an incremental backup, but no tools
> whatsoever to do anything useful with it once you've got it.

As stated previously, Marco is writing a tool called pg_restorebackup (the
prototype in Python has been already posted) to be included in the core. I
am in Australia now and not in the office so I cannot confirm it, but I am
pretty sure he had already written it and was about to send it to the list.

He's been trying to find more data - see the one that he's sent - in order
to convince that even a file-based approach is useful.

I think that's 100% unacceptable.

I agree, that's why pg_restorebackup written in C is part of this patch.
See: https://wiki.postgresql.org/wiki/Incremental_backup

> Users need to be able to manipulate
> PostgreSQL backups using either standard operating system tools or
> tools provided with PostgreSQL. Some people may prefer to use
> something like repmgr or pitrtools or omniptr in addition, but that
> shouldn't be a requirement for incremental backup to be usable.
>

Not at all. I believe those tools will have to use pg_basebackup and
pg_restorebackup. If they want to use streaming replication protocol they
will be responsible to make sure that - if the protocol changes - they
adapt their technology.

Agile development is good, but that does not mean you can divide a big
> project into arbitrarily small chunks. At some point the chunks are
> too small to be sure that the overall direction is right, and/or
> individually useless.
>

The goal has always been to provide "file-based incremental backup". I can
assure that this has always been our compass and the direction to follow.

I repeat that, using pg_restorebackup, this patch will transparently let
users benefit from incremental backup even when it will be moved to an
internal block-level logic. Users will continue to execute pg_basebackup
and pg_restorebackup, ignoring that with - for example 9.5 - it is
file-based (saving between 50-70% of space and time) of block level - for
example 9.6.

My proposal is that Marco provides pg_restorebackup according to the
initial plan - a matter of hours/days.

Cheers,
Gabriele