Re: pg_start_backup: file has vanished from pg_subtrans/

On 8/7/15 7:01 AM, Leo Baltus wrote:
> While backing up some postgresql-8.4.2 instances using
> pg_start_backup()/pg_stop_backup()
>
> We noticed that sometimes rsync complains:
>
> file has vanished: "............./data/pg_subtrans/3A37"
> rsync warning: some files vanished before they could be transferred (code 24) at main.c(1039) [sender=3.0.6

It's perfectly normal for Postgres to remove files during a backup. If
a table is dropped then the files storing that table will be deleted.
Files can also be removed from pg_subtrans once they are no longer needed.

> We already have excluded pgsql_tmp/ pg_xlog/ and pg_stat_tmp/ so I was
> tempted to exclude pg_subtrans/ as well. However some googling seems to
> suggest that this might invalidate the backup:

I would recommend only excluding pg_xlog as recommended in the Postgres
docs.

> https://groups.google.com/forum/#!msg/pgbarman/zQxg6pOXkZ8/AmTmVtUA3dEJ
>
> I am not sure what to believe, either
> - my backup was invalid and I have to take measures to prevent this
> (and file a bug report)

Your backup should be good so long as you've captured all the WAL.

> - I can ignore this *and* can safely exclude pg_trans/ from my backups.
> - there is some other workaround...

Do *not* exclude pg_subtrans.

You may also want to look at backup software such as pgBackRest. It
understands that files can be removed during backup and does not emit
warnings when it happens (though info messages are still logged
depending on your log level).

--
-David
david(at)pgmasters(dot)net