Re: First set of OSDL Shared Mem scalability results, some

On 10/8/2004 10:10 PM, Christopher Browne wrote:

> josh(at)agliodbs(dot)com (Josh Berkus) wrote:
>> I've been trying to peg the "sweet spot" for shared memory using
>> OSDL's equipment. With Jan's new ARC patch, I was expecting that
>> the desired amount of shared_buffers to be greatly increased. This
>> has not turned out to be the case.
>
> That doesn't surprise me.

Neither does it surprise me.

>
> My primary expectation would be that ARC would be able to make small
> buffers much more effective alongside vacuums and seq scans than they
> used to be. That does not establish anything about the value of
> increasing the size buffer caches...

The primary goal of ARC is to prevent total cache eviction caused by
sequential scans. Which means it is designed to avoid the catastrophic
impact of a pg_dump or other, similar access in parallel to the OLTP
traffic. It would be much more interesting to see how a half way into a
2 hour measurement interval started pg_dump affects the response times.

One also has to take a closer look at the data of the DBT2. What amount
of that 32GB is high-frequently accessed, and therefore a good thing to
live in the PG shared cache? A cache significantly larger than that
doesn't make sense to me, under no cache strategy.

Jan

>
>> This result is so surprising that I want people to take a look at it
>> and tell me if there's something wrong with the tests or some
>> bottlenecking factor that I've not seen.
>
> I'm aware of two conspicuous scenarios where ARC would be expected to
> _substantially_ improve performance:
>
> 1. When it allows a VACUUM not to throw useful data out of
> the shared cache in that VACUUM now only 'chews' on one
> page of the cache;
>
> 2. When it allows a Seq Scan to not push useful data out of
> the shared cache, for much the same reason.
>
> I don't imagine either scenario are prominent in the OSDL tests.
>
> Increasing the number of cache buffers _is_ likely to lead to some
> slowdowns:
>
> - Data that passes through the cache also passes through kernel
> cache, so it's recorded twice, and read twice...
>
> - The more cache pages there are, the more work is needed for
> PostgreSQL to manage them. That will notably happen anywhere
> that there is a need to scan the cache.
>
> - If there are any inefficiencies in how the OS kernel manages shared
> memory, as their size scales, well, that will obviously cause a
> slowdown.

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #