Re: [PATCH] pg_basebackup: progress report max once per second

On Thu, Jan 9, 2014, at 20:05, Magnus Hagander wrote:
> On Thu, Nov 14, 2013 at 10:27 AM, Mika Eloranta <mel(at)ohmu(dot)fi> wrote:
>> On 13 Nov 2013, at 20:51, Mika Eloranta <mel(at)ohmu(dot)fi> wrote:
>>
>>
>>
> Prevent excessive progress reporting that can grow to gigabytes
>>
> of output with large databases.
>>
>>
>> Same patch as an attachment.
>
> Would it not make more sense to instead store the last number printed, and only print it if the percentage has changed? AIUI with this patch we still print the same thing on top of itself if it takes >1 second to get 1% further.
>
> (Except for verbose mode - but if you're asking for verbose mode, you are *asking* to get lots of output) 

(re-sent response as I used the wrong sender address previously, sorry
about the dupe)

Printing out progress periodically is IMHO slightly better as the
interactive user can see that there is some progress (e.g. by pressing
enter for a new empty console line) during a huge basebackup operation.

The original problem I wanted to address was that I had a daemon
watching over the basebackup process and reading its output to make sure
that the basebackup is proceeding properly. It also wrote all the output
to a logfile for postmortem analysis. The log file grew to a very big
size (multiple gigabytes due to the progress prints). With my patch the
log was only a few kilos, without any negative effects. The excessive
progress reporting can also be an issue in an interactive session over a
slow network (mobile), hogging both time and bandwidth.

Cheers,

Mika