Re: Introducing a new linux readahead framework

On Thu, Apr 20, 2006 at 11:31:47PM -0500, Jim C. Nasby wrote:
> > In adaptive readahead, the context based method may be of particular
> > interest to postgresql users. It works by peeking into the file cache
> > and check if there are any history pages present or accessed. In this
> > way it can detect almost all forms of sequential / semi-sequential read
> > patterns, e.g.
> > - parallel / interleaved sequential scans on one file
> > - sequential reads across file open/close
> > - mixed sequential / random accesses
> > - sparse / skimming sequential read
> >
> > It also have methods to detect some less common cases:
> > - reading backward
> > - seeking all over reading N pages
>
> Are there any ways to inform the kernel that you either are or aren't
> doing a sequential read? It seems that in some cases it would be better

This call will disable readahead totally for fd:
posix_fadvise(fd, any, any, POSIX_FADV_RANDOM);

This one will reenable it:
posix_fadvise(fd, any, any, POSIX_FADV_NORMAL);

This one will enable readahead _and_ set max readahead window to
2*max_readahead_kb:
posix_fadvise(fd, any, any, POSIX_FADV_SEQUENTIAL);

> to bypass a bunch of tricky logic trying to determine that it's doing a
> sequential read. A sequential scan in PostgreSQL would be such a case.

You do not need to worry about the detecting `overhead' on sequential
scans :) The adaptive readahead framework has a fast code path(the
stateful method) to handle normal sequential reads, the detection of
which is really trivial.

> The opposite example would be an index scan of a highly uncorrelated
> index, which would produce mostly random reads from the table. In that
> case, reading ahead probably makes very little sense, though your logic
> might have a better idea of the access pattern than PostgreSQL does.

As for the index scans, the failsafe code path(i.e. the context based
one) will normally be used, and it does have a little overhead in
looking up the page cache(about 0.4% more CPU time). However, the
penalty of random disk access is so large that if ever it helps
reducing a small fraction of disk accesses, you wins.

Thanks,
Wu