Re: Move unused buffers to freelist

On Friday, June 28, 2013 6:20 PM Robert Haas wrote:
On Fri, Jun 28, 2013 at 12:52 AM, Amit Kapila <amit(dot)kapila(at)huawei(dot)com> wrote:
>> Currently it wakes up based on bgwriterdelay config parameter which is by
>> default 200ms, so you means we should
>> think of waking up bgwriter based on allocations and number of elements left
>> in freelist?

> I think that's what Andres and I are proposing, yes.

>> As per my understanding Summarization of points raised by you and Andres
>> which this patch should address to have a bigger win:
>>
>> 1. Bgwriter needs to be improved so that it can help in reducing usage count
>> and finding next victim buffer
>> (run the clock sweep and add buffers to the free list).

> Check.

>> 2. SetLatch for bgwriter (wakeup bgwriter) when elements in freelist are
>> less.

>Check. The way to do this is to keep a variable in shared memory in
>the same cache line as the spinlock protecting the freelist, and
>update it when you update the free list.

>> 3. Split the workdone globallock (Buffreelist) in StrategyGetBuffer
>> (a spinlock for the freelist, and an lwlock for the clock sweep).

>Check.

>> 4. Separate processes for writing dirty buffers and moving buffers to
>> freelist

> I think this part might be best pushed to a separate patch, although I
> agree we probably need it.

>> 5. Bgwriter needs to be more aggressive, logic based on which it calculates
>> how many buffers it needs to process needs to be improved.

> This is basically overlapping with points already made. I suspect we
> could just get rid of bgwriter_delay, bgwriter_lru_maxpages, and
> bgwriter_lru_multiplier altogether. The background writer would just
> have a high and a low watermark. When the number of buffers on the
> freelist drops below the low watermark, the allocating backend sets
> the latch and bgwriter wakes up and begins adding buffers to the
> freelist. When the number of buffers on the free list reaches the
> high watermark, the background writer goes back to sleep. Some
> experimentation might be needed to figure out what values are
> appropriate for those watermarks. In theory this could be a
> configuration knob, but I suspect it's better to just make the system
> tune it right automatically.

Do you think it will be sufficient to just wake bgwriter when the buffers in freelist drops
below low watermark, how about it's current job of flushing dirty buffers?

I mean to ask that if for some scenario where there are sufficient buffers in freelist, but most
other buffers are dirty, will delaying flush untill number of buffers fall below low watermark is okay.

>> 6. There can be contention around buffer mapping locks, but we can focus on
>> it later
>> 7. cacheline bouncing around the buffer header spinlocks, is there anything
>> we can do to reduce this?

> I think these are points that we should leave for the future.

with Regards,
Amit Kapila.