Re: Load distributed checkpoint

From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "Jim C(dot) Nasby" <jim(at)nasby(dot)net>, "ITAGAKI Takahiro" <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>, <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Load distributed checkpoint
Date: 2006-12-08 17:01:47
Message-ID: 4579461A.EE98.0025.0@wicourts.gov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches

>>> On Fri, Dec 8, 2006 at 10:43 AM, in message
<6439(dot)1165596207(at)sss(dot)pgh(dot)pa(dot)us>,
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> I wonder whether it would be feasible to teach the bgwriter to get
more
> aggressive as the time for the next checkpoint approaches? Writes
> issued early in the interval have a much higher probability of being
> wasted (because the page gets re- dirtied later).

But wouldn't the ones written earlier stand a much better chance of
being scheduled for a physical write before the fsync? They aren't ALL
re-dirtied. In our environment, we seem to be getting a lot written
before the fsync with our current settings.

> But maybe that just
> reduces to what Takahiro- san already suggested, namely that
> checkpoint- time writes should be done with the same kind of
scheduling
> the bgwriter uses outside checkpoints. We still have the problem
that
> the real I/O storm is triggered by fsync() not write(), and we don't
> have a way to spread out the consequences of fsync().

We don't have a way to force writes before the fsync, but early writes
to the file system encourages it. After reading this thread, I'm
tempted to nudge our settings a little higher -- especially the
percentages. How much overhead is there in checking whether buffer
pages are dirty?

-Kevin

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message jorge alberto 2006-12-08 17:04:29 picksplit algorithm
Previous Message jorge alberto 2006-12-08 16:53:23 #define GEVHDRSZ ( offsetof(GistEntryVector, vector[0]) ) explanation please

Browse pgsql-patches by date

  From Date Subject
Next Message Kevin Grittner 2006-12-08 17:55:09 Re: Load distributed checkpoint
Previous Message Tom Lane 2006-12-08 16:43:27 Re: Load distributed checkpoint