Re: WAL Rate Limiting

On 2014-01-16 17:47:51 +0200, Heikki Linnakangas wrote:
> On 01/16/2014 05:39 PM, Andres Freund wrote:
> >On 2014-01-16 10:35:20 -0500, Tom Lane wrote:
> >>I think possibly a more productive approach to this would be to treat
> >>it as a session-level GUC setting, rather than hard-wiring it to affect
> >>certain commands and not others.
> >
> >Do you see a reasonable way to implement this generically for all
> >commands? I don't.
>
> In suitable safe places, check if you've written too much WAL, and sleep if
> so. Call it CHECK_FOR_WAL_BUDGET(), along the lines of
> CHECK_FOR_INTERRUPTS(), but called less frequently. Or maybe
> vacuum_delay_point() is a better analogy. Hopefully you don't need to
> sprinkle them in too many places to be useful.

That'd be most places doing XLogInsert() if you want to be
consistent. Each needing to be analyzed whether we're blocking important
resources, determining where in the callstack we can do the
CHECK_FOR_WAL_BUDGET().

I don't think the the add power by allowing bulk DML to be metered is
worth the increased implementation cost.

I think the usecases that would want this for DML probably also wan this
to work for unlogged, temp tables. That'd require a much more generic
resource control framework.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services