Re: 7.3.1 New install, large queries are slow

"Josh Berkus" <josh(at)agliodbs(dot)com> writes:
>>> shared_buffers = 131072
>>
>> Yipes! Try about a tenth that much. Or less.

> Why? He has 4GB RAM on the machine.

I think a gig of shared buffers is overkill no matter what.

One reason not to crank up shared_buffers "just because you can" is that
there are operations (such as CHECKPOINT) that have to scan through all
the buffers, linearly. I don't *think* any of these are in
performance-critical paths, but nonetheless you're wasting CPU. I trust
the kernel to manage a huge number of buffers efficiently more than I
trust Postgres.

There's another issue, which is somewhat platform-dependent; I'm not
sure if it applies to whatever OS Roman is using. But if you have a
machine where SysV shared memory is not locked into RAM, then a huge
shared buffer arena creates the probability that some of it will be
touched seldom enough that the kernel will decide to swap it out. When
that happens, you *really* pay through the nose --- a page that you
might have been able to get from kernel cache without incurring I/O will
now certainly cost you I/O to touch. It's even worse if the buffer
contained a dirty page at the time it was swapped out --- now that page
is going to require being read back in and written out again, a net cost
of three I/Os where there should have been one. Bottom line is that
shared_buffers should be kept small enough that the space all looks like
a hot spot to the kernel's memory allocation manager.

In short, I believe in keeping shared_buffers relatively small --- one
to ten thousand seems like the right ballpark --- and leaving the kernel
to allocate the rest of RAM as kernel disk cache.

I have been thinking recently about proposing that we change the factory
default shared_buffers to 1000, which if this line of reasoning is
correct would eliminate the need for average installations to tune it.
The reason the default is 64 is that on some older Unixen, the default
SHMMAX is only one meg --- but it's been a long time since our default
shared memory request was less than a meg anyway, because of bloat in
other components of shared memory. It's probably time to change the
default to something more reasonable from a performance standpoint, and
put some burden on users of older Unixen to either reduce the setting
or fix their SHMMAX parameter.

regards, tom lane