From: | Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com> |
---|---|
To: | Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov> |
Cc: | Greg Smith <greg(at)2ndquadrant(dot)com>, pgsql-performance(at)postgresql(dot)org |
Subject: | Re: Linux memory zone reclaim |
Date: | 2012-07-30 17:09:35 |
Message-ID: | CAOR=d=2R5VKL2nEff3KpTWrLoZStyfOnAOwfeNC3b67ecSFJnw@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-performance |
On Mon, Jul 30, 2012 at 10:43 AM, Kevin Grittner
<Kevin(dot)Grittner(at)wicourts(dot)gov> wrote:
> node distances:
> node 0 1 2 3
> 0: 10 11 11 11
> 1: 11 10 11 11
> 2: 11 11 10 11
> 3: 11 11 11 10
>
> When considering a hardware purchase, it might be wise to pay close
> attention to how "far" a core may need to go to get to the most
> "distant" RAM.
I think the zone_reclaim gets turned on with a high ratio. If the
inter node costs were the same, and the intranode costs dropped in
half, zone reclaim would likely get turned on at boot time.
I had something similar in a 48 core system but if I recall correctly
the matrix was 8x8 and the cost differential was much higher.
The symptoms I saw was that a very hard working db, on a 128G machine
with about 95G as OS / kernel cache, would slow to a crawl with kswapd
working very hard (I think it was kswapd) after a period of 1 to 3
weeks. Note that actual swap in and out wasn't all that great by
vmstat. The same performance hit happened on a similar machine used
as a file server after a similar period of warm up.
The real danger here is that the misbehavior can take a long time to
show up, and from what I read at the time, the performance gain for
any zone reclaim = 1 was minimal for a file or db server, and more in
line for a large virtual machine farm, with a lot of processes chopped
into sections small enough to fit in one node's memory and not need a
lot of access from another node. Anything that relies on the OS to
cache is likely not served by zone reclaim = 1.
From | Date | Subject | |
---|---|---|---|
Next Message | Bruce Momjian | 2012-07-30 17:26:58 | Re: Linux memory zone reclaim |
Previous Message | Kevin Grittner | 2012-07-30 16:43:36 | Re: Linux memory zone reclaim |