Re: WAL Rate Limiting

On 01/16/2014 11:52 PM, Tom Lane wrote:
> Heikki Linnakangas <hlinnakangas(at)vmware(dot)com> writes:
>> On 01/16/2014 05:39 PM, Andres Freund wrote:
>>> Do you see a reasonable way to implement this generically for all
>>> commands? I don't.
>
>> In suitable safe places, check if you've written too much WAL, and sleep
>> if so. Call it CHECK_FOR_WAL_BUDGET(), along the lines of
>> CHECK_FOR_INTERRUPTS(), but called less frequently. Or maybe
>> vacuum_delay_point() is a better analogy. Hopefully you don't need to
>> sprinkle them in too many places to be useful.
>
> We probably don't really need to implement it for "all" commands; I think
> everyone can agree that, say, ALTER TABLE RENAME COLUMN isn't going to
> emit enough WAL to really matter. My point was that we should try to hit
> everything that potentially *could* generate lots of WAL, rather than
> arbitrarily deciding that some are utility commands and some are not.

Then you land up needing a configuration mechanism to control *which*
commands get affected, too, to handle the original use case of "I want
to rate limit vaccuum and index rebuilds, while everything else runs
full tilt".

Or do you propose to just do this per-session? If so, what about autovacuum?

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services