Re: [PATCHES] ARC Memory Usage analysis

Curt Sampson <cjs(at)cynic(dot)net> writes:

> On Tue, 26 Oct 2004, Greg Stark wrote:
>
> > I see mmap or O_DIRECT being the only viable long-term stable states. My
> > natural inclination was the former but after the latest thread on the subject
> > I suspect it'll be forever out of reach. That makes O_DIRECT And a Postgres
> > managed cache the only real choice. Having both caches is just a waste of
> > memory and a waste of cpu cycles.
>
> I don't see why mmap is any more out of reach than O_DIRECT; it's not
> all that much harder to implement, and mmap (and madvise!) is more
> widely available.

Because there's no way to prevent a write-out from occurring and no way to be
notified by mmap before a write-out occurs, and Postgres wants to do its WAL
logging at that time if it hasn't already happened.

> But if using two caches is only costing us 1% in performance, there's
> not really much point....

Well firstly it depends on the work profile. It can probably get much higher
than we saw in that profile if your work load is causing more fresh buffers to
be fetched.

Secondly it also reduces the amount of cache available. If you have 256M of
ram with about 200M free, and 40Mb of ram set aside for Postgres's buffer
cache then you really only get 160Mb. It's costing you 20% of your cache, and
reducing the cache hit rate accordingly.

Thirdly the kernel doesn't know as much as Postgres about the load. Postgres
could optimize its use of cache based on whether it knows the data is being
loaded by a vacuum or sequential scan rather than an index lookup. In practice
Postgres has gone with ARC which I suppose a kernel could implement anyways,
but afaik neither linux nor BSD choose to do anything like it.

--
greg