Re: Read-ahead and parallelism in redo recovery

Florian G. Pflug wrote:
> Greg Stark wrote:
> > Florian G. Pflug wrote:
> >> The same holds true for index scans, though. Maybe we can find a
> >> solution that benefits both cases - something along the line of a
> >> bgreader process
> > I posted a patch to do readahead for bitmap index scans using
> > posix_fadvise. Experiments showed it works great on raid arrays on
> > Linux. Solaris will need to use libaio though which I haven't tried
> > yet.
> Cool! I'd like to try it out - is that patch available in the pg-patches
> archives?
>
> > Doing it for normal index scans is much much harder. You can
> > readahead a single page by using the next pointer if it looks like
> > you'll need it. But I don't see a convenient way to get more than
> > that.
> I was thinking that after reading a page from the index, the backend
> could post a list of heap pages referenced from that index page to the
> shmem. A background process would repeatedly scan that list, and load
> those pages into the buffer cache.

Agreed. Lots of database do the index/heap readahead via threads --- I
think we will probably use a separate read-ahead process that knows more
about all the concurrent reads and the tablespaces involved.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://postgres.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +