Re: skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

On Wed, Jun 10, 2015 at 8:29 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> On Mon, Jun 8, 2015 at 12:09 AM, Michael Paquier
> <michael(dot)paquier(at)gmail(dot)com> wrote:
> >> Recently, one of our customers has had a basebackup fail because pg_log
> >> contained files that were >8GB:
> >> FATAL: archive member "pg_log/postgresql-20150119.log" too large for
> tar format
> >>
> >> I think pg_basebackup should also skip pg_log entries, as it does for
> >> pg_stats_temp and pg_replslot, etc. I've attached a patch along those
> >> lines for discussion.
> >
> > And a recent discussion about that is this one:
> >
> http://www.postgresql.org/message-id/82897A1301080E4B8E461DDAA0FFCF142A1B2660@SYD1216
> > Bringing the point: some users may want to keep log files in a base
> > backup, and some users may want to skip some of them, and not only
> > pg_log. Hence we may want more flexibility than what is proposed here.
>
> That seems pretty thin. If you're taking a base backup, your goal is
> to create a standby.

Mine goal isn't that. My goal is to have a consistent backup without
having to shut down the server to take a cold one, or having to manually
juggle the pg_start_backup, etc. commands. I do occasionally use it start
up a standby for training/testing purposes, but mostly it is for D-R (in
which I would rather have the logs) and for cloning test/dev/QA
environments (in which case I go delete the logs if I don't want them)

> Copying logs is in no way an integral part of
> that, and we would not copy them if they were stored outside the data
> directory. If we accept the proposal that this needs to be more
> complicated, will we also accept a proposal to make pg_basebackup
> include relevant files from /var/log when the PostgreSQL logs are
> stored there?
>

I think it is pretty intuitive that if you have your logs go to pg_log,
they get backed up with the other pg_ stuff, and if you change it go
elsewhere, then you need to handle it yourself.

Cheers,

Jeff