Re: Experimental patch for inter-page delay in VACUUM

Ang Chin Han wrote:

> Christopher Browne wrote:
>
>> Centuries ago, Nostradamus foresaw when "Stephen"
>> <jleelim(at)xxxxxxx(dot)com> would write:
>>
>>> As it turns out. With vacuum_page_delay = 0, VACUUM took 1m20s (80s)
>>> to complete, with vacuum_page_delay = 1 and vacuum_page_delay = 10,
>>> both VACUUMs completed in 18m3s (1080 sec). A factor of 13 times!
>>> This is for a single 350 MB table.
>>
>> While it is unfortunate that the minimum quanta seems to commonly be
>> 10ms, it doesn't strike me as an enormous difficulty from a practical
>> perspective.
>
> If we can't lower the minimum quanta, we could always vacuum 2 pages
> before sleeping 10ms, effectively sleeping 5ms.

Right, I think this is what Jan has done already.

> What would be interesting would be pg_autovacuum changing these values
> per table, depending on current I/O load.
>
> Hmmm. Looks like there's a lot of interesting things pg_autovacuum can
> do:
> 1. When on low I/O load, running multiple vacuums on different,
> smaller tables on full speed, careful to note that these vacuums will
> increase the I/O load as well.
> 2. When on high I/O load, vacuum big, busy tables slowly.

I'm not sure how practacle any of this is. How will pg_autovacuum
surmise the current I/O load of the system? Keeping in mind that
postgres is not the only cause of I/O. Also, the optimum delay for a
long running vacuum might change dramatically while it's running. If
there is a way to judge the current I/O load, it might be better for
vacuum to auto-tune itself while it's running, perhaps based on some
hints given to it by pg_autovacuum or manually by a user. For example,
a delay hint of 0 should always be zero no matter what. A delay hint of
1 will scale up slower than a delay hint of 2 which would scale up
slower than 5 etc....

Of course this is all wild conjecture if there isn't an easy way to
surmise the system I/O load. Thoughts?