Re: pg_rewind proposed scope and interface changes

On Tue, Sep 12, 2017 at 11:52 PM, Chris Travers
<chris(dot)travers(at)adjust(dot)com> wrote:
> Additionally the wal, xact, timestamp and logical directories must be
> processed in some way.

To what does the term "logical directories" refer to?

> * if --wal=sync the directories are processed the way they are today
> * if --wal=clear then the contents of the directories are cleared and
> replication is assumed to be used to bring the system up after. Note this
> will need to come with warning about the need for replication slots.

Hm. I am not sure in what --wal=clear is helpful. Keeping around WAL
segments from the point of the last checkpoint where WAL forked up to
the point where WAL has forked is helpful, because you don't need to
copy again those WAL segments, be they come from an archive or from
streaming. Copying a set of WAL segments during the rewind of the new
timeline is helpful as well because you don't need to do the copy
again. One configuration where this is helpful is that there is
already an archive local to the target server available with the
segments of the new timeline available.

> Base, global, pg_tablespace
>
> With
> pg_wal, pg_xact, pg_commit_ts, pg_logical added if wal strategy is set to
> sync.

Skipping some directories in a way similar to what a base backup does
would be nicer I think. We already have a list of those in
basebackup.c in the shape of excludeDirContents and excludeFiles. I
think that it would be a good idea to export those into a header that
pg_rewind could include, and refer to in order to exclude them when
fetching a set of files. At the end of the day, a rewind is a kind of
base backup in itself, and this patch would already serve well a lot
of people.

Having on top of that a way to exclude a wanted set of files and the
log directory (for example: should we look at log_directory and
exclude it from the fetched paths if it is not an absolute path?),
which is smart enough to take care of not removing paths critical for
a rewind like anything in base/, then you are good to go with a
full-blown tool that I think would serve the purposes you are looking
for.
--
Michael