Re: [Lsf-pc] Linux kernel impact on PostgreSQL performance

The issue with O_DIRECT is actually a much more general issue ---
namely, database programmers that for various reasons decide they
don't want to go down the O_DIRECT route, but then care about
performance. PostgreSQL is not the only database which is had this
issue.

There are two papers at this year's FAST conference about the "Journal
of Journal" (JoJ) problem, which has been triggered by the use of SQLite on
android handsets, and its write patterns, some of which some folks
(including myself) have characterized as "abusive". (As in, when the
database developer says to the kernel developer, "Doctor, doctor, it
hurts when I do that...")

The program statement for JoJ was introduced in last year's Usenix ATC
conference, I/O Stack Optimizations for Smartphones[1]

[1] https://www.usenix.org/conference/atc13/technical-sessions/presentation/jeong

The high order bit is what's the right thing to do when database
progammers come to kernel engineers saying, we want to do <FOO> and
the performance sucks. Do we say, "Use O_DIRECT, dummy", not
withstanding Linus's past comments on the issue? Or do we have some
general design principles that we tell database engineers that they
should do for better performance, and then all developers for all of
the file systems can then try to optimize for a set of new API's, or
recommended ways of using the existing API's?

Surely the wrong answer is that we do things which encourage people to
create entire new specialized file systems for different databases.
The f2fs file system was essentially created because someone thought
it was easier to create a new file system from sratch instad of trying
to change how SQLite or some other existing file system works.
Hopefully we won't have companies using MySQL and PostgreSQL deciding
they need their own mysqlfs and postgresqlfs! :-)

Cheers,

- Ted