Re: pg_dump why no indicator of completion

I backup some databases every night using pg_dump.  From a bash script
initiated by cron, the way God intended them to be run.  And I check the
return code, like every good production shell script should.

I back up other databases using pgBackRest (also from a bash script
initiated by cron, the way God intended them to be run).  And I check the
return code, like every good production shell script should (even though it
prints a message saying "Finished").

Because who in their right mind manually checks log files every morning,
when a shell script will gladly email you if something went wrong? *Make the
computer work for you!!!*

On 5/1/23 09:31, richard coleman wrote:
> Ron,
>
> That seems to be a very odd stance.  PostgreSQL and it's various utilities
> are and have been used in very large commercial operations for decades.
> pg_dump successfully backs up databases from 1MB to 10TB and beyond.  The
> only issue I am having is for some inexplicable reason the devs haven't
> included a status message upon completion.
>
> If you don't think that pg_dump is fit for purpose, then what do you
> propose using in its stead?
>
> thanks,
> rik.
>
> On Mon, May 1, 2023 at 10:07 AM Ron <ronljohnsonjr(at)gmail(dot)com> wrote:
>
> That's like complaining that a Toyota Camry isn't fit for hauling 10
> cubic yards of sand.
>
>
> On 5/1/23 08:55, richard coleman wrote:
>> Ron,
>>
>> Are you writing that pg_dump is unfit for purpose and that I should
>> be using a commercial backup solution instead?
>>
>> rik.
>>
>> On Mon, May 1, 2023 at 9:43 AM Ron <ronljohnsonjr(at)gmail(dot)com> wrote:
>>
>>
>>
>> On 5/1/23 07:56, richard coleman wrote:
>>> Ron,
>>> So what you are saying is that I have to write a shell script
>>> around pg_dump to catch the exit code of pg_dump since the
>>> developers of pg_dump didn't have pg_dump simply write an exit
>>> message?
>>>
>>> Since multiple pg_dump commands are often run at the same
>>> command prompt, and they can take hours, if not days to run,
>>
>> Running a days-long pg_dump at any time except special occasions
>> isn't very wise.  Especially from a command prompt.
>>
>> Binary backup/restore programs are /much/ faster (if for no other
>> reason than they let you do incremental and differential backups).
>>
>>> and there are a myriad of other commands the will be run in the
>>> interim, any exit code generated by pg_dump with be lost in the
>>> flotsam and jetsam of the multitude of exit codes created by
>>> every other command run between the calling of pg_dump and it's
>>> eventual termination. It could have completed successfully,
>>> crashed, been killed, etc.
>>>
>>> Or am I mistaken?
>>>
>>> thanks,
>>> rik.
>>>
>>> On Sun, Apr 30, 2023 at 11:28 PM Ron <ronljohnsonjr(at)gmail(dot)com>
>>> wrote:
>>>
>>> On 4/30/23 19:18, richard coleman wrote:
>>>> Hi all,
>>>>
>>>> I've been working with pg_dump and one thing that's always
>>>> struck me as strange is that there never seems to be an
>>>> indication in the log that the pg_dump process completed
>>>> successfully.
>>>>
>>>> For example, I've been running a bunch of backups with
>>>> pg_dump in directory mode with multiple jobs. When I come
>>>> back several hours later there are no pg_dump processes
>>>> running.  The log is filled with:
>>>>
>>>> dumping contents of foo
>>>> finished item 123456 of TABLE DATA bar
>>>> etc.
>>>>
>>>>
>>>> Other than trying to restore the multi terabyte database
>>>> somewhere else is there any way to know that it actually
>>>> finished successfully?
>>>>
>>>> Why doesn't pg_dump add a line in the output like:
>>>>
>>>> pg_backup finished
>>>>
>>>> when it's completed successfully?
>>>>
>>>> This seems like a terrible oversight.
>>>>
>>>> Am I missing something obvious?
>>>
>>> "They" expect you to do the Unix Thing and check $?. Thus,
>>> that's what I do; any non-zero value generates an email with
>>> a scary subject line, the specifics of which are based on
>>> the exact code value.  I'm not near my work computer;
>>> otherwise I'd share them with you.
>>>
>>> --
>>> Born in Arizona, moved to Babylonia.
>>>
>>
>> --
>> Born in Arizona, moved to Babylonia.
>>
>
> --
> Born in Arizona, moved to Babylonia.
>

--
Born in Arizona, moved to Babylonia.