Re: [Lsf-pc] Linux kernel impact on PostgreSQL performance

On Tue 14-01-14 10:04:16, Robert Haas wrote:
> On Tue, Jan 14, 2014 at 5:00 AM, Jan Kara <jack(at)suse(dot)cz> wrote:
> > I thought that instead of injecting pages into pagecache for aging as you
> > describe in 3), you would mark pages as volatile (i.e. for reclaim by
> > kernel) through vrange() syscall. Next time you need the page, you check
> > whether the kernel reclaimed the page or not. If yes, you reload it from
> > disk, if not, you unmark it and use it.
> >
> > Now the aging of pages marked as volatile as it is currently implemented
> > needn't be perfect for your needs but you still have time to influence what
> > gets implemented... Actually developers of the vrange() syscall were
> > specifically looking for some ideas what to base aging on. Currently I
> > think it is first marked - first evicted.
>
> This is an interesting idea but it stinks of impracticality.
> Essentially when the last buffer pin on a page is dropped we'd have to
> mark it as discardable, and then the next person wanting to pin it
> would have to check whether it's still there. But the system call
> overhead of calling vrange() every time the last pin on a page was
> dropped would probably hose us.
>
> *thinks*
>
> Well, I guess it could be done lazily: make periodic sweeps through
> shared_buffers, looking for pages that haven't been touched in a
> while, and vrange() them. That's quite a bit of new mechanism, but in
> theory it could work out to a win. vrange() would have to scale well
> to millions of separate ranges, though. Will it?
It is intented to be rather lightweight so I believe milions should be
OK. But I didn't try :).

> And a lot depends on whether the kernel makes the right decision about
> whether to chunk data from our vrange() vs. any other page it could have
> reclaimed.
I think the intent is to reclaim pages in the following order:
used once pages -> volatile pages -> active pages, swapping

Honza
--
Jan Kara <jack(at)suse(dot)cz>
SUSE Labs, CR