Re: Experimental patch for inter-page delay in VACUUM

Bruce Momjian wrote:

> Tom Lane wrote:
>> Andrew Sullivan <andrew(at)libertyrms(dot)info> writes:
>> > On Sun, Nov 02, 2003 at 01:00:35PM -0500, Tom Lane wrote:
>> >> real traction we'd have to go back to the "take over most of RAM for
>> >> shared buffers" approach, which we already know to have a bunch of
>> >> severe disadvantages.
>>
>> > I know there are severe disadvantages in the current implementation,
>> > but are there in-principle severe disadvantages?
>>
>> Yes. For one, since we cannot change the size of shared memory
>> on-the-fly (at least not portably), there is no opportunity to trade off
>> memory usage dynamically between processes and disk buffers. For
>> another, on many systems shared memory is subject to being swapped out.
>> Swapping out dirty buffers is a performance killer, because they must be
>> swapped back in again before they can be written to where they should
>> have gone. The only way to avoid this is to keep the number of shared
>> buffers small enough that they all remain fairly "hot" (recently used)
>> and so the kernel won't be tempted to swap out any part of the region.
>
> Agreed, we can't resize shared memory, but I don't think most OS's swap
> out shared memory, and even if they do, they usually have a kernel

We can't resize shared memory because we allocate the whole thing in one
big hump - which causes the shmmax problem BTW. If we allocate that in
chunks of multiple blocks, we only have to give it a total maximum size
to get the hash tables and other stuff right from the beginning. But the
vast majority of memory, the buffers themself, can be made adjustable at
runtime.

Jan

> configuration parameter to lock it into kernel memory. All the old
> unixes locked the shared memory into kernel address space and in fact
> this is why many of them required a kernel recompile to increase shared
> memory. I hope the ones that have pagable shared memory have a way to
> prevent it --- at least FreeBSD does, not sure about Linux.
>
> Now, the disadvantages of large kernel cache, small PostgreSQL buffer
> cache is that data has to be transfered to/from the kernel buffers, and
> second, we can't control the kernel's cache replacement strategy, and
> will probably not be able to in the near future, while we do control our
> own buffer cache replacement strategy.
>
> Looking at the advantages/disadvantages, a large shared buffer cache
> looks pretty good to me.
>

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #