Re: Getting rid of AtEOXact Buffers (was Re: [Testperf-general] Re: [PERFORM] First set of OSDL Shared Memscalability results, some wierdness ...)

Jan Wieck <JanWieck(at)Yahoo(dot)com> writes:
> I realized that StrategyDirtyBufferList currently wasts a lot of time by
> first scanning over all the buffers that haven't even been hit since
> it's last call and neither have been dirty last time (and thus, are at
> the beginning of the list and can't be dirty anyway). If we would have a
> way to give it a smart "point in the list to start scanning" ...

I don't think it's true that they *can't* be dirty.

(1) Buffers are marked dirty when released, whereas they are moved to
the fronts of the lists when acquired.

(2) the cntxDirty bit can be set asynchronously to any BufMgrLock'd
operation.

But it sure seems like we are doing more work than we really need to.

One idea I had was for the bgwriter to collect all the dirty pages up to
say halfway on the LRU lists, and then write *all* of these, not just
the first N, over as many rounds as are needed. Then go back and call
StrategyDirtyBufferList again to get a new list. (We don't want it to
write every dirty buffer this way, because the ones near the front of
the list are likely to be dirtied again right away. But certainly we
could write more than 1% of the dirty buffers without getting into the
area of the recently-used buffers.)

There isn't any particularly good reason for this to share code with
checkpoint-style BufferSync, btw. BufferSync could just as easily scan
the buffers linearly, since it doesn't matter what order it writes them
in. So we could change StrategyDirtyBufferList to stop as soon as it's
halfway up the LRU lists, which would save at least a few cycles.

regards, tom lane