Re: StrategyGetBuffer optimization, take 2

On Mon, Aug 19, 2013 at 5:02 PM, Jeff Janes <jeff(dot)janes(at)gmail(dot)com> wrote:
>
> My concern is how we can ever move this forward. If we can't recreate
> it on a test system, and you probably won't be allowed to push
> experimental patches to the production system....what's left?
>
> Also, if the kernel is introducing new scheduling bottlenecks, are we
> just playing whack-a-mole by trying to deal with it in PG code?

I think I can rest on the basis that there is no good reason to sit
and spin on a buffer during clock sweep -- even if I have to whittle
the patch down to the '1 liner' variant that trylocks the buffer
header. I also like the idea of getting rid of the freelist lock
completely but I think those ideas can be tested in isolation. Really
though, the acid-test is going to be exhaustive performance testing.

> Stepping back a bit, have you considered a connection pooler to
> restrict the number of simultaneously active connections? It wouldn't
> be the first time that that has alleviated the symptoms of these
> high-RAM kernel bottlenecks. (If we are going to play whack-a-mole,
> might as well use a hammer that already exists and is well tested)

Certainly. It was the very first thing I suggested upon hearing about
this problem. Unfortunately, when these kinds of things happen in
high scale, high load databases on serious hardware improvising
pgbouncer is simply not possible without first going through extensive
performance and application compatibility testing.

merlin