Re: [PROPOSAL] VACUUM Progress Checker.

On 2016/03/15 3:41, Robert Haas wrote:
> On Sat, Mar 12, 2016 at 7:49 AM, Amit Langote <amitlangote09(at)gmail(dot)com> wrote:
>> Instead, the attached patch adds a IndexBulkDeleteProgressCallback
>> which AMs should call for every block that's read (say, right before a
>> call to ReadBufferExtended) as part of a given vacuum run. The
>> callback with help of some bookkeeping state can count each block and
>> report to pgstat_progress API. Now, I am not sure if all AMs read 1..N
>> blocks for every vacuum or if it's possible that some blocks are read
>> more than once in single vacuum, etc. IOW, some AM's processing may
>> be non-linear and counting blocks 1..N (where N is reported total
>> index blocks) may not be possible. However, this is the best I could
>> think of as doing what we are trying to do here. Maybe index AM
>> experts can chime in on that.
>>
>> Thoughts?
>
> Well, I think you need to study the index AMs and figure this out.

OK. I tried to put calls to the callback in appropriate places, but
couldn't get the resulting progress numbers to look sane. So I ended up
concluding that any attempt to do so is futile unless I analyze each AM's
vacuum code carefully to be able to determine in advance the max bound on
the count of blocks that the callback will report. Anyway, as you
suggest, we can improve it later.

> But I think for starters you should write a patch that reports the following:
>
> 1. phase
> 2. number of heap blocks scanned
> 3. number of heap blocks vacuumed
> 4. number of completed index vac cycles
> 5. number of dead tuples collected since the last index vac cycle
> 6. number of dead tuples that we can store before needing to perform
> an index vac cycle
>
> All of that should be pretty straightforward, and then we'd have
> something we can ship. We can add the detailed index reporting later,
> when we get to it, perhaps for 9.7.

OK, I agree with this plan. Attached updated patch implements this.

Thanks,
Amit