Re: high io BUT huge amount of free memory

On 2013-04-24 08:39:09 -0500, Shaun Thomas wrote:
> The memory pressure code in Linux is extremely fucked up. I can't find it
> right now, but the memory management algorithm makes some pretty ridiculous
> assumptions once you pass half memory usage, regarding what is in active and
> inactive cache.
>
> I hate to rant, but it gets clearer to me every day that Linux is optimized
> for desktop systems, and generally only kinda works for servers. Once you
> start throwing vast amounts of memory, CPU, and processes at it though,
> things start to get unpredictable.

> That all goes back to my earlier threads that disabling process autogrouping
> via the kernel.sched_autogroup_enabled setting, magically gave us 20-30%
> better performance. The optimal setting for a server is clearly to disable
> process autogrouping, and yet it's enabled by default, and strongly
> advocated by Linus himself as a vast improvement.

> I get it. It's better for desktop systems. But the LAMP stack alone has
> probably a couple orders of magnitude more use cases than Joe Blow's Pentium
> 4 in his basement. Yet it's the latter case that's optimized for.

IIRC there have been some scalability improvements to that code.

> Servers are getting shafted in a lot of cases, and it's actually starting to
> make me angry.

Uh. Ranting can be rather healthy thing every now and then and it good
for the soul and such. But. Did you actually try reporting those issues?
In my experience the mm and scheduler folks are rather helpful if they
see you're actually interested in fixing a problem. I have seen rants
about this topic on various pg lists for the last months but I can't
remember seeing mails on lkml about it. How should they fix what they
don't know about?

You know, before Robert got access to the bigger machine we *also* had
some very bad behaviour on them. And our writeout mechanism/buffer
acquisition mechanism still utterly sucks there. But that doesn't mean
we don't care.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services