Re: Why we are going to have to go DirectIO

On Wed, Dec 4, 2013 at 9:31 PM, Jonathan Corbet <corbet(at)lwn(dot)net> wrote:

> > I also wasn't exaggerating the reception I got when I tried to talk
> > about IO and PostgreSQL at LinuxCon and other events. The majority of
> > Linux hackers I've talked to simply don't want to be bothered with
> > PostgreSQL's performance needs, and I've heard similar things from my
> > collegues at the MySQL variants. Greg KH was the only real exception.
> >
> > Heck, I went to a meeting of filesystem geeks at LinuxCon and the main
> > feedback I received, from Linux FS developers (Chris and Ted), was
> > "PostgreSQL should implement its own storage and use DirectIO, we don't
> > know why you're even trying to use the Linux IO stack."
>
> I think you're talking to the wrong people. Nothing you've described is a
> filesystem problem; you're contending with memory management problems.
> Chris and Ted weren't helpful because there's actually little they can do
> to help you. I would be happy to introduce you to some people who would be
> more likely to take your problems to heart.
>
> Mel Gorman, for example, is working on putting together a set of MM
> benchmarks in the hopes of quantifying changes and catching regressions
> before new code is merged. He's one of the people who has to deal with
> performance regressions when they show up in enterprise kernels, and I get
> the sense he'd rather do less of that.
>
> Perhaps even better: the next filesystem, storage, and memory management
> summit is March 24-25. A session on your pain points there would bring in
> a substantial portion of the relevant developers at all levels. LSFMM
> is arguably the most productive kernel event I see over the course of a
> year; it's where I would go first to make progress on this issue. I'm not
> an LSFMM organizer, but I would be happy to work to make such a session
> happen if somebody from the PostgreSQL community wanted to be there.
>

I think that's an excellent idea. If one of our developers could find the
time to attend that, I think that could be very productive. While I'm not
on the funds team, I'd definitely vote for funding such participation out
of community funds if said developer can't do it on his own.

But it should definitely be a developer with interest and skills in that
particular area as well of course :) So don't think I'm proposing myself, I
definitely am not :)

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/