Re: WAL Rate Limiting

On Thu, Feb 20, 2014 at 12:07 PM, Alvaro Herrera
<alvherre(at)2ndquadrant(dot)com> wrote:
> I think "bulk" (maintenance) operations should legitimately configured
> separately from regular UPDATE etc operations. For the latter I think
> it's pretty reasonable to assume that users are going to want to tweak
> the GUC for each individual callsite in the application, rather than
> wholesale in postgresql.conf.

There's an argument to be made for that design, but the discussion
upthread doesn't support the contention that the patch makes a clean
and well-defined division between those things. The initial patch
left VACUUM out on the dubious theory that since we have an existing
system to limit its throughput by pages dirtied we don't need a way to
limit the rate at which it generates WAL, and left as an open question
whether this out to apply to COPY and CREATE TABLE AS SELECT (but
apparently not UPDATE or DELETE, or for that matter just plain SELECT,
when it does a lot of pruning). A subsequent version of the patch
changed the list of commands affected, but it's not at all clear to me
that we have something as tidy as "only commands where X and never
commands where Y".

More broadly, three committers (myself, Tom, Heikki) expressed the
desire for this to be made into a more generic mechanism, and two of
those people (myself and Tom) said that this was too late for 9.4 and
should be postponed to 9.5. Then a month went by and Greg Stark
showed up and made noises about pushing this forward as-is. So I
don't want it to be forgotten that those objections were made and have
not been withdrawn. I'm not dead set against changing my position on
this patch, but that should happen because of work to improve the
patch - which was last posted on January 17th and did not at that time
even include the requested and agreed fix to measure the limit in MB/s
rather than some odd unit - not because of the simple passage of time.
If anything, the passage of 5 weeks without meaningful progress ought
to strengthen rather than weaken the argument that this is far too
late for this release.

Of course, if we're talking about 9.5, then disregard the previous
paragraph. But in that case perhaps we could postpone this until we
don't have 40+ patches needing review and a dozen marked ready for
committer.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company