Re: pg_dump why no indicator of completion

From: Tim <timfosho(at)gmail(dot)com>
To: Ron <ronljohnsonjr(at)gmail(dot)com>
Cc: pgsql-admin(at)lists(dot)postgresql(dot)org
Subject: Re: pg_dump why no indicator of completion
Date: 2023-05-02 01:29:50
Message-ID: CAKhLO5iRHh6JoxBJyMV8RkEYWfjiE_qBhiUu1YTkLPAhjWwpuQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

This has been a very entertaining thread to read through. Thankyou.

> I will admit that I haven't tried writing custom binary outputs yet,
typically using either the builtin custom format or directory format.

If you really are working with Muli-TB databases (kinda hard to believe in
the first place, doing a backup that takes many hours with pg_dump will
cause your WAL folder to fill up), then its time to learn some more mature
tools.

pg-dump made sense maybe a decade ago. As recommended above, nowadays
people use *pgbackrest* and *pgbarman*.

> Since multiple pg_dump commands are often run at the same command prompt,
and they can take hours, if not days to run,
> and there are a myriad of other commands the will be run in the interim,
any exit code generated by pg_dump with be lost
> in the flotsam and jetsam of the multitude of exit codes created by
every other command run between the calling of pg_dump
> and it's eventual termination. It could have completed successfully,
crashed, been killed, etc.

Your workflow sounds completely mystifying.

> then I am going to be forced to create a wrapper to capture and emit the
status around the pg_dump executable.
> If I need to do that, so will every other user who needs to know the exit
status and runs pg_dump in a typical linux
> workflow.

I'm sorry, but If putting your multi terabyte backup process into a 3 line
script that checks the output and maybe emailing
you or just appending to the logfile seems like a crazy idea, you're going
to be completely floored once you learn a few extra basic scripting
patterns in the world of Linux.

You're basically saying you're entirely constrained to triggering processes
from the terminal by hand (please use a cronjob) and due to that
insurmountable limitation, you also wont learn how an && works.

> Seems like a terrible waste having untold numbers of users having to
constantly reinvent the wheel.

You might be overstating things a bit

On Mon, May 1, 2023 at 12:15 PM Ron <ronljohnsonjr(at)gmail(dot)com> wrote:

> On 5/1/23 10:50, Scott Ribe wrote:
>
> On May 1, 2023, at 9:25 AM, David G. Johnston <david(dot)g(dot)johnston(at)gmail(dot)com> <david(dot)g(dot)johnston(at)gmail(dot)com> wrote:
>
> The arguments about "the Unix way" seem to have forgotten the point about being silent except in cases of exception, and this level of detail already violates that premise.
>
> It would be a reasonable convenience. I think some of the pushback is from the framing that not having it is a defect.
>
>
> Exactly. It's a convenience, the lack of which is easily worked around.
>
> What would *really* be useful are timestamps (both start and completion
> of dump) and an easy association of ".dat numbers" with table names.
>
> --
> Born in Arizona, moved to Babylonia.
>

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message richard coleman 2023-05-02 02:42:29 Re: pg_dump why no indicator of completion
Previous Message Ron 2023-05-01 16:15:02 Re: pg_dump why no indicator of completion