Re: [PROPOSAL] VACUUM Progress Checker.

On Mon, Aug 10, 2015 at 1:36 PM, Rahila Syed <rahilasyed90(at)gmail(dot)com> wrote:
> Hello,
>
>>Say, 6 bigint counters, 6 float8
>>counters, and 3 strings up to 80 characters each. So we have a
>>fixed-size chunk of shared memory per backend, and each backend that
>>wants to expose progress information can fill in those fields however
>>it likes, and we expose the results.
>>This would be sorta like the way pg_statistic works: the same columns
>>can be used for different purposes depending on what estimator will be
>>used to access them.
>
> After thinking more on this suggestion, I came up with following generic
> structure which can be used to store progress of any command per backend in
> shared memory.
>
> Struct PgBackendProgress
> {
> int32 *counter[COMMAND_NUM_SLOTS];
> float8 *counter_float[COMMAND_NUM_SLOTS];
>
> char *progress_message[COMMAND_NUM_SLOTS];
> }
>
> COMMAND_NUM_SLOTS will define maximum number of slots(phases) for any
> command.
> Progress of command will be measured using progress of each phase in
> command.
> For some command the number of phases can be singular and rest of the slots
> will be NULL.
>
> Each phase will report n integer counters, n float counters and a progress
> message.
> For some phases , any of the above fields can be NULL.
>
> For VACUUM , there can 3 phases as discussed in the earlier mails.
>
> Phase 1. Report 2 integer counters: heap pages scanned and total heap pages,
> 1 float counter: percentage_complete and progress message.
> Phase 2. Report 2 integer counters: index pages scanned and total index
> pages(across all indexes) and progress message.
> Phase 3. 1 integer counter: heap pages vacuumed.
>
> This structure can be accessed by statistics collector to display progress
> via new view.

I have one question about this.

When we're in Phase2 or 3, don't we need to report the number of total
page scanned or percentage of how many table pages scanned, as well?
As Robert said, Phase2(means lazy_vacuum_index here) and 3(means
lazy_vacuum_heap here) could be called whenever lazy_scan_heap fills
up the maintenance_work_mem. And phase 3 could be called at the end of
scanning single page if table doesn't have index.
So if vacuum progress checker reports the only current phase
information, we would not be able to know where we are in now.

Regards,

--
Masahiko Sawada