Re: base backup and tar problems with disappearing files.

Sorry Alban , For the late update.

Yep John,

I am talking about the tar'ring of pgdatadir only excluding the pg_xlog
dir.
We have set up our full backup system in accordance to the admin guide.
Even the guide puts forward the limitation of tar in producing
distinguishing exit codes.

My doubt at this moment is , Is it normal to expect files disappearing from
the
pgdatadir during the course of taking base backup ? I can think about the
temp
sorting files disappearing but i am not sure what could cause data files
disappear
like example given below:

tar: /mnt/disk1/pgdatadir/base/16399/861272781: Cannot stat: No such file or
directory

If the above incidence is normal then i would only worry about making tar
not to worry
about the disappearing files.

The reason of putting this question is that the line in one of the later
paras
of section "24.3.2" says

"Some file system backup tools emit warnings or errors if the files they are
trying to
copy change while the copy proceeds." , it only says about "change" not
disappearance.

Since i have deep respect for the excellent documentation quality of
PostgreSQL project
i read and interpret it by words.

Warm Regds
-mallah.

On Tue, Mar 15, 2011 at 1:42 PM, John R Pierce <pierce(at)hogranch(dot)com> wrote:

> On 03/15/11 12:30 AM, Alban Hertroys wrote:
>
>> On 15 Mar 2011, at 7:46, Alban Hertroys wrote:
>>
>> On 15 Mar 2011, at 3:06, Rajesh Kumar Mallah wrote:
>>>
>>> Dear Friends,
>>>>
>>>> While taking online basebackup we ignore tar exit codes of 1 .
>>>> However under certain circumstances tar exits we code '2' which
>>>> stands for 'Fatal Errors' . Eg in case of "Cannot stat: No such file or
>>>> directory"
>>>> encountered while taking backup of the pgdatadir . My question is
>>>> can we ignore such errors of "vanishing files" ? is it normal ?
>>>> I think the situation is arising because some table which were created
>>>> before start of backup were dropped during the backup. But that is
>>>> quite normal also.
>>>>
>>>
>>> You should probably exclude the PG data directories from your file-system
>>> backups, there isn't much point in backing them up anyway.
>>>
>>
>> I should refine that a bit...
>>
>> A file-system level backup backs up the files in a sequential order, while
>> the database writes it's transactions in them in a pattern that's much
>> closer to random order. As a result of that, your file-system backup is
>> likely to contain the database files in an inconsistent state.
>> If you subsequently try to recover from that backup, you rely on the
>> ability of the database to recover from that inconsistent state. Postgres is
>> pretty good at recovering, but there's no guarantee it will succeed. It's
>> probably a bad idea to rely on that for your backups.
>>
>> Instead, for backing up your database, use one of the strategies outlined
>> in the fine manual. Those are reliable.
>>
>
>
> um, I assumed from the original post that he was talking about taking a
> base backup in preparation for setting up WAL replication, presumably
> preceded by a call to pg_start_backup(), etc...
>
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>