Re: Linux max on shared buffers?

On Thu, 11 Jul 2002, Martijn van Oosterhout wrote:

> Yeah, I thought of the writeback issue also. I was thinking that you
> might have to keep the current shared memory scheme for written pages
> but use mmap for reading them in. Since the number of written pages
> is much smaller than the number of read pages, you can avoid a lot of
> double buffering.

Actually, come to think of it, since we write the entire new page
to the WAL, we'd have to copy the page anyway. So this basically
solves the problem; you mark the page as having a "to-write" copy,
copy it, modify it, write the log file, and copy the new page back
to the mmap buffer. Not too difficult at all, really.

If we ever did stop writing the entire page to WAL, it's not too
difficult anyway, becuase we just have to do the actually write to
the page *after* the log entry is on stable storage, as I pointed
out, which means keeping a record of the change to be written. But
we have that record of course, because it's exactly what we just
wrote to the log. And we also, of course, have logic to take that
record and apply it to the table...

> ...and possibly cause portability
> problems, since that could make assumptions about how buffers, shared
> memory and mmaps are shared.

No, portability problems are very unlikely. We already use shared
memory and it works. Mmap has very well defined behaviour and, in
my experience, works very consistently across all modern systems.

cjs
--
Curt Sampson <cjs(at)cynic(dot)net> +81 90 7737 2974 http://www.netbsd.org
Don't you know, in this new Dark Age, we're all light. --XTC