Re: DB cache size strategies

Martha Stewart called it a Good Thing when pgsql(at)bluepolka(dot)net ("Ed L.") wrote:
> On Tuesday February 10 2004 3:48, scott.marlowe wrote:
>> On Tue, 10 Feb 2004, Ed L. wrote:
>> > Interesting. Why leave very large tables to the kernel instead
>> > of the db cache? Assuming a dedicated DB server and a DB smaller
>> > than available RAM, why not give the DB enough RAM to get the
>> > entire DB into the DB cache? (Assuming you have the RAM).
>>
>> Because the kernel is more efficient (right now) at caching large
>> data sets.
>>
>> With the ARC cache manager that will likely wend it's way into 7.5,
>> it's quite a likely possibility that postgresql will be able to
>> efficiently handle a larger cache, but it will still be a shared
>> memory cache, and those are still usually much slower than the
>> kernel's cache.
>
> Hmmm. Others have asserted/assumed they'd be roughly equivalent.
> It'd be interesting to see some real data measuring the performance
> of the shared mem cache vs. kernel cache. Anyone know of existing
> benchmarks?

The problem with the shared memory cache is that there seems to be
some variation from OS to OS as to how efficiently the system can
manage shared memory as the amount of it grows.

The same is commonly true for various sorts of resources:

- On Linux, for a long time, you didn't want to have a lot of files in
one directory as it did a linked-list scan to look for files.
ReiserFS specifically planned an improvement to this by replacing
the linked list with a b-tree.

- Linux and Solaris are conspicuously different in their process
scheduling models. With Solaris, performance suffers if you have
a great deal of processes; the architecture encourages use of
threading to get good performance. In contrast, Linux 2.6 has
what they call an "O(1)" process scheduler where having lots of
processes does not substantially slow down OS performance.

The benchmarks have been a bit limited thus far; people haven't found
that having much more than 10000 buffer pages to be particularly
helpful. And there has been some indication that sizes larger than
that can slow things down. And that may not be a constant across OS
implementations...
--
"cbbrowne","@","cbbrowne.com"
http://www3.sympatico.ca/cbbrowne/rdbms.html
Rules of the Evil Overlord #151. "I will not set myself up as a
god. That perilous position is reserved for my trusted lieutenant."
<http://www.eviloverlord.com/>