Re: [Proposal] Progress bar for pg_dump/pg_restore

Hi, andres

Thank you for your comment, and sorry for late response.

> The question is how to actually get useful estimates. As there's no
> progress report for indvidiual COPY and CREATE INDEX commands you'll, in
> many cases, have very irregular progress updates. In many many cases
> most of the time is spent on a very small subset of the total objects.

When dumping, I think number of tuples can be got from pg_class.reltuples, therefore I want pg_dump to run "select reltuples" to get it, and then pg_dump will calculate estimated time to execute "COPY FROM" command in getting each tuples.

For restoring, I think it's better to record above information (number of tuples) into pg_dump file to estimate time to restore tables.

And, I also understood your concern about "CREATE INDEX", but we have no way to get progress information of "CREATE INDEX".
At present, I think it may be good to refer to the same time as estimated time to execute "COPY TO".
But it's better to get information from pg_stat_activity which is proposed at other thread from Anzai-san as following.

http://www.postgresql.org/message-id/116262CF971C844FB6E793F8809B51C6EA6E21@BPXM02GP.gisp.nec.co.jp

How about your opinion?

regards,
--
Taiki Kondo

-----Original Message-----
From: pgsql-hackers-owner(at)postgresql(dot)org [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Andres Freund
Sent: Friday, June 12, 2015 10:48 PM
To: Taiki Kondo
Cc: pgsql-hackers(at)postgresql(dot)org; Akio Iwaasa
Subject: Re: [HACKERS] [Proposal] Progress bar for pg_dump/pg_restore

Hi,

On 2015-06-12 12:45:50 +0000, Taiki Kondo wrote:
> Design & API
> ------------
> When pg_dump / pg_restore is running, progress bar and estimated time to finish is shown on screen like following.
>
>
> =========> (50%) 15:50
>
> The bar ("=>" in above) and percentage value ("50%" in above) show percentage of progress, and the time ("15:50" in above) shows estimated time to finish.
> (This percentage is the ratio for the whole processing.)
>
> Percentage and time are calculated and shown for every 1 second.
>
> In pg_dump, the information, which is required for calculating percentage and time, is from pg_class.
>
> In pg_restore, to calculate the same things, I want to record total amount of command lines into pg_dump file, thus I would like to add a new element to "Archive" structure.
> (This means that version number of archive format is changed.)

The question is how to actually get useful estimates. As there's no progress report for indvidiual COPY and CREATE INDEX commands you'll, in many cases, have very irregular progress updates. In many many cases most of the time is spent on a very small subset of the total objects.

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org) To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers