Re: [Lsf-pc] Linux kernel impact on PostgreSQL performance

On Tue, Jan 14, 2014 at 5:00 AM, Jan Kara <jack(at)suse(dot)cz> wrote:
> I thought that instead of injecting pages into pagecache for aging as you
> describe in 3), you would mark pages as volatile (i.e. for reclaim by
> kernel) through vrange() syscall. Next time you need the page, you check
> whether the kernel reclaimed the page or not. If yes, you reload it from
> disk, if not, you unmark it and use it.
>
> Now the aging of pages marked as volatile as it is currently implemented
> needn't be perfect for your needs but you still have time to influence what
> gets implemented... Actually developers of the vrange() syscall were
> specifically looking for some ideas what to base aging on. Currently I
> think it is first marked - first evicted.

This is an interesting idea but it stinks of impracticality.
Essentially when the last buffer pin on a page is dropped we'd have to
mark it as discardable, and then the next person wanting to pin it
would have to check whether it's still there. But the system call
overhead of calling vrange() every time the last pin on a page was
dropped would probably hose us.

*thinks*

Well, I guess it could be done lazily: make periodic sweeps through
shared_buffers, looking for pages that haven't been touched in a
while, and vrange() them. That's quite a bit of new mechanism, but in
theory it could work out to a win. vrange() would have to scale well
to millions of separate ranges, though. Will it? And a lot depends
on whether the kernel makes the right decision about whether to chunk
data from our vrange() vs. any other page it could have reclaimed.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company