Re: Readme of Buffer Management seems to have wrong sentence

>>Do we have any evidence that this is actually a problem which needs to be
solved?

I don't have very clear evidence, didn't generate any profiling report
still. However in the scenario
Where there are many buffers in the list and backend has to traverse it,
there should be reasonable work under lock.

>>You would need to lock once to put the buffer on, and again to take it
off. That increases the traffic through a >>contended lock, rather than
decreasing it.

But it shall reduce time of backends which traverse the buffer list as in
some cases they will get the buffer directly from freelist. I understand
that some new contention will be created but wanted to see if the amount it
has reduced can out weight the new contention that gets created.

>>The problem is that there is no way to test it in enough different
situations to convince people it would be a general >>improvement.
I think in this whole idea main point was to have scenarios which can prove
all the points are win. This is same point raised by Greg as well. My first
thought was to generate heavy contention on shared buffers will be able to
show up but I still need to think more on it.
Any more ideas and suggestions to generate the scenarios?

-----Original Message-----
From: Jeff Janes [mailto:jeff(dot)janes(at)gmail(dot)com]
Sent: Wednesday, May 23, 2012 11:48 PM
To: Amit Kapila
Cc: Tom Lane; Robert Haas; PostgreSQL-development
Subject: Re: [HACKERS] Readme of Buffer Management seems to have wrong
sentence

On Wed, May 23, 2012 at 8:36 AM, Amit Kapila <amit(dot)kapila(at)huawei(dot)com> wrote:
>>>And besides
>>>if the decrements are decoupled from the allocation requests it's no
>>>longer obvious that the algorithm is even an approximation of LRU.
>
> I was trying to highlight that we can do the clocksweep in bgwriter and
keep
> the backends logic as it is currently.
> The core idea is that it will reduce the work of backends and chances of
> them to get the free buffer early than currently will be more.

Do we have any evidence that this is actually a problem which needs to
be solved? I've seen plenty of evidence that contention for the lock
can be a problem, but no evidence that the amount of work done under
the lock, once it is obtained, is a problem.

> Some of the other ideas about it which I have discussed are
> 1. moving clean buffers to freelist when any found by bgwriter or
> checkpoint. This is to get the buffers early by backends without going
into
> clock sweep algorithm.

You would need to lock once to put the buffer on, and again to take it
off. That increases the traffic through a contended lock, rather than
decreasing it. So unless this is coupled with reducing the lock
strength, I don't see how it can win.

>
> 2. having multiple lists of hot and cold buffers and backends will find
the
> buffers in following order if the required buffer is not already there
>   a. freelist
>   b. cold list
>   c. hot list
>
> 3. try to experiment with different values of usage count under heavy load
> scenarios and check does it make any difference.

One thing I wanted to play with is having newly read buffers get a
usage count of 0 rather than 1. The problem is that there is no way
to test it in enough different situations to convince people it would
be a general improvement.

Cheers,

Jeff