Re: [PATCHES] Load Distributed Checkpoints, revised patch

Michael Paesold wrote:
> Heikki Linnakangas wrote:
>> Here's an updated WIP version of the LDC patch. I just spreads the
>> writes, that achieves the goal of smoothing the checkpoint I/O spikes.
>> I think sorting the writes etc. is interesting but falls in the
>> category of further development and should be pushed to 8.4.
>
> Why do you think so? Is it too much risk to adapt the sorted writes? The
> numbers shown by ITAGAKI Takahiro looked quite impressive, at least for
> large shared_buffers configurations. The reactions where rather
> positive, too.

Well, it is a very recent idea, and it's not clear that it's a win under
all circumstances. Adopting that would need more testing, and at this
late stage I'd like to just wrap up what we have and come back to this
idea for 8.4.

If someone performs the tests with different hardware and workloads, I
would be willing to consider it; the patch is still at an early stage
but it's very isolated change and should therefore be easy to review.
But if someone has the hardware and time perform those tests, I'd like
them to perform more testing with just LDC first. As Josh pointed out,
it would be good to test it with more oddball workloads, all the tests
I've done this far have been with DBT-2.

> In general, I am hoping that this patch, together with "Automatic
> adjustment of bgwriter_lru_maxpages" will finally make default
> postgresql configurations experience much less impact from check points.
> For my tast, postgresql has recently got way to many nobs which one must
> tweak by hand... I welcome any approach on auto-tuning (and auto vacuum!).

Sure, but that's another topic.

> Patch status says "waiting on update from author":
> http://archives.postgresql.org/pgsql-patches/2007-04/msg00331.php
> Any updates on this?

No. I'm not actually clear what we're waiting for and from whom; I know
I haven't had the time to review that in detail yet. IIRC we've seen two
very similar patches in the discussions, one from Itagaki-san, and one
from Greg Smith. I'm not sure which one of them we should use, they both
implement roughly the same thing. But the biggest thing needed for that
patch is testing with different workloads.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com