Re: postgres files in use not staying in linux file cache

On 06/05/2014 04:32 PM, Brio wrote:

> So here's where I'm stuck. How can reading a file not leave it in the
> Linux cache? I'd expect it to enter the inactive list (which is about
> 80GB), so I'd expect another 80GB would need to be read before it would
> be its turn to be evicted.... which should take a long time if my
> maximum read speed is 100MB/s.

So here's the thing. The Linux page reclamation code is *extremely
broken* in everything before 3.11. Take a look at this, then realize
that this is *only one patch* from several that target the memory
manager weightings:

http://linux-kernel.2935.n7.nabble.com/patch-v2-0-3-mm-improve-page-aging-fairness-between-zones-nodes-td696105.html

This is especially true of the 3.2 kernel you're using. It's extremely
aggressive about ageing pages out of memory when there's high memory
pressure from frequent disk reads. Chances of promotion into the active
set is dismal, so you end up with a constant churn between inactive and
disk. Worse, if you kick it hard enough by having too many PostgreSQL
backends using memory, it'll actively purge the active set while still
failing to promote the inactive set.

The dev that linked me to this patch said he tested it against 3.10,
meaning it probably went into 3.11 or 3.12. So I personally wouldn't
trust anything before 3.13. :p

Since you're using Ubuntu 12.04, I strongly suggest upgrading your core
to 12.04.4 and apply the linux-generic-lts-saucy pseudo-package to at
least get onto the 3.11 instead. The 3.2 kernel is pants-on-head
retarded; we've had a lot more luck with 3.8 and above.

Cheers!

--
Shaun Thomas
OptionsHouse, LLC | 141 W. Jackson Blvd. | Suite 800 | Chicago IL, 60604
312-676-8870
sthomas(at)optionshouse(dot)com

______________________________________________

See http://www.peak6.com/email_disclaimer/ for terms and conditions related to this email