From: | Simon Riggs <simon(at)2ndQuadrant(dot)com> |
---|---|
To: | PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | WAL Rate Limiting |
Date: | 2014-01-15 02:20:13 |
Message-ID: | CA+U5nMLfxBgHQ1VLSeBHYEMjHXz_OHSkuFdU6_1quiGM0RNKEg@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
We've discussed previously the negative impact of large bulk
operations, especially wrt WAL writes. Patch here allows maintenance
operations to have their WAL generation slowed down as a replication
lag prevention feature.
I believe there was originally intended to be some work on I/O rate
limiting, but that hasn't happened and is in some ways orthogonal to
this patch and we will likely eventually want both.
Single new parameter works very similarly to vacuum_cost_delay
wal_rate_limit_delay = Xms
slows down CLUSTER, VACUUM FULL, ALTER TABLE (rewrite & set
tablespace), CREATE INDEX
so basically same things we optimise WAL for and the same places where
we honour maintenance_work_mem
(discuss: should we add COPY, CTAS etc also?)
(discuss: do we need another parameter to specify "cost"? Currently
patch uses "sleep every 64kB of WAL")
VACUUM is not included, since we already have controls for that -
honouring two controls would be complex and weird.
Uses GetCurrentTransactionWALVolume() patch, which is included within
the patch to make it easier to review as a whole.
Technically, we can't simply wait before/after WAL inserts because
these typically occur while holding buffer locks. So we need to put
the waits at a higher level, notably in safe places that currently do
CHECK_FOR_INTERRUPTS(). Doing that during query execution might make
locking of blocks for nested loops joins much worse.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachment | Content-Type | Size |
---|---|---|
wal_rate_limiting.v1.patch | application/octet-stream | 14.6 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Dave Chinner | 2014-01-15 02:25:56 | Re: [Lsf-pc] Linux kernel impact on PostgreSQL performance |
Previous Message | Michael Paquier | 2014-01-15 02:07:41 | Re: CREATE TABLESPACE WITH |