Re: posix_fadvsise in base backups

Hi Greg,

On Sunday, September 25, 2011 03:25:50 AM Greg Stark wrote:
> On Sat, Sep 24, 2011 at 4:16 PM, Magnus Hagander <magnus(at)hagander(dot)net>
wrote:
> > I was assuming the kernel was smart enough to read this as "*this*
> > process is not going to be using this file anymore", not "nobody in
> > the whole machine is going to use this file anymore". And the process
> > running the base backup is certainly not going to read it again.
> >
> > But that's a good point - do you know if that is the case, or does it
> > mandate more testing?
> It's not the case on Linux. I used to use DONTNEED to flush pages from
> cache before running a benchmark. I verified with mincore that the
> pages were actually getting removed from cache. Sometimes there was
> the occasional straggler but nearly all got flushed and after a second
> or third pass the stragglers were gone too.
Not sure what exactly is "not the case on Linux". Your answer could be read in
a way that the fadvise/DONTNEED adheres to some sort of refcounting scheme
(which it afaik does not) or that it doesn't.

> In case you're wondering, this was because using /proc/.../drop_caches
> caused flaky benchmarks. My theory was that it was causing pages of
> the executable to trigger page faults in the middle of the benchmark.
That should be easily possible to rule out by preloading the
applications+libraries?
I think there were plans to teach the dynamic linker to enforce doing so, but
I am not sure they were ever folloowed through.

Andres