Re: Experimental patch for inter-page delay in VACUUM

scott.marlowe wrote:
> On Tue, 4 Nov 2003, Tom Lane wrote:
>
> > Jan Wieck <JanWieck(at)Yahoo(dot)com> writes:
> > > What still needs to be addressed is the IO storm cause by checkpoints. I
> > > see it much relaxed when stretching out the BufferSync() over most of
> > > the time until the next one should occur. But the kernel sync at it's
> > > end still pushes the system hard against the wall.
> >
> > I have never been happy with the fact that we use sync(2) at all. Quite
> > aside from the "I/O storm" issue, sync() is really an unsafe way to do a
> > checkpoint, because there is no way to be certain when it is done. And
> > on top of that, it does too much, because it forces syncing of files
> > unrelated to Postgres.
> >
> > I would like to see us go over to fsync, or some other technique that
> > gives more certainty about when the write has occurred. There might be
> > some scope that way to allow stretching out the I/O, too.
> >
> > The main problem with this is knowing which files need to be fsync'd.
>
> Wasn't this a problem that the win32 port had to solve by keeping a list
> of all files that need fsyncing since Windows doesn't do sync() in the
> classical sense? If so, then could we use that code to keep track of the
> files that need fsyncing?

Yes, I have that code from SRA. They used threading, so they recorded
all the open files in local memory and opened/fsync/closed them for
checkpoints. We have to store the file names in a shared area, perhaps
an area of shared memory with an overflow to a disk file.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073