Re: PATCH: Exclude unlogged tables from base backups

Hi,

On 2017-12-12 18:04:44 -0500, David Steele wrote:
> On 12/12/17 5:52 PM, Andres Freund wrote:
> > On 2017-12-12 17:49:54 -0500, David Steele wrote:
> > > Including unlogged relations in base backups takes up space and is wasteful
> > > since they are truncated during backup recovery.
> > >
> > > The attached patches exclude unlogged relations from base backups except for
> > > the init fork, which is required to recreate the main fork during recovery.
> >
> > How do you reliably identify unlogged relations while writes are going
> > on? Without locks that sounds, uh, nontrivial?
>
> I don't think this is an issue. If the init fork exists it should be OK if
> it is torn since it will be recreated from WAL.

I'm not worried about torn pages.

> If the forks are written out of order (i.e. main before init), which is
> definitely possible, then I think worst case is some files will be backed up
> that don't need to be. The main fork is unlikely to be very large at that
> point so it doesn't seem like a big deal.
>
> I don't see this as any different than what happens during recovery. The
> unlogged forks are cleaned / re-inited before replay starts which is the
> same thing we are doing here.

It's quite different - in the recovery case there's no other write
activity going on. But on a normally running cluster the persistence of
existing tables can get changed, and oids can get recycled. What
guarantees that between the time you checked for the init fork the table
hasn't been dropped, the oid reused and now a permanent relation is in
its place?

Greetings,

Andres Freund