Re: [Lsf-pc] Linux kernel impact on PostgreSQL performance

On Sun, Jan 19, 2014 at 03:37:37AM +0200, Marti Raudsepp wrote:
> On Wed, Jan 15, 2014 at 5:34 AM, Jim Nasby <jim(at)nasby(dot)net> wrote:
> > it's very common to create temporary file data that will never, ever, ever
> > actually NEED to hit disk. Where I work being able to tell the kernel to
> > avoid flushing those files unless the kernel thinks it's got better things
> > to do with that memory would be EXTREMELY valuable
>
> Windows has the FILE_ATTRIBUTE_TEMPORARY flag for this purpose.
>
> ISTR that there was discussion about implementing something analogous
> in Linux when ext4 got delayed allocation support, but I don't think
> it got anywhere and I can't find the discussion now. I think the
> proposed interface was to create and then unlink the file immediately,
> which serves as a hint that the application doesn't care about
> persistence.

You're thinking about O_TMPFILE, which is for making temp files that
can't be seen in the filesystem namespace, not for preventing them
from being written to disk.

I don't really like the idea of overloading a namespace directive to
have special writeback connotations. What we are getting into the
realm of here is generic user controlled allocation and writeback
policy...

> Postgres is far from being the only application that wants this; many
> people resort to tmpfs because of this:
> https://lwn.net/Articles/499410/

Yes, we covered the possibility of using tmpfs much earlier in the
thread, and came to the conclusion that temp files can be larger
than memory so tmpfs isn't the solution here. :)

Cheers,

Dave.
--
Dave Chinner
david(at)fromorbit(dot)com