Re: WAL Rate Limiting

On 1/14/14, 8:20 PM, Simon Riggs wrote:
> We've discussed previously the negative impact of large bulk
> operations, especially wrt WAL writes. Patch here allows maintenance
> operations to have their WAL generation slowed down as a replication
> lag prevention feature.
>
> I believe there was originally intended to be some work on I/O rate
> limiting, but that hasn't happened and is in some ways orthogonal to
> this patch and we will likely eventually want both.
>
> Single new parameter works very similarly to vacuum_cost_delay
>
> wal_rate_limit_delay = Xms

From a replication standpoint, I think it would be a lot nicer if this also (or instead) considered the replication delay we were currently experiencing.

Related to that... it would be extremely useful to us if we could similarly rate limit certain backend operations, for the purpose of not overloading the database or replication. That's fairly unrelated except that sometimes you might want to WAL rate limit a backend operation, besides just COPY.
--
Jim C. Nasby, Data Architect jim(at)nasby(dot)net
512.569.9461 (cell) http://jim.nasby.net