Performance tuning on RedHat Enterprise Linux 3

From: "David Esposito" <pgsql-general(at)esposito(dot)newnetco(dot)com>
To: <pgsql-general(at)postgresql(dot)org>
Subject: Performance tuning on RedHat Enterprise Linux 3
Date: 2004-12-06 14:08:02
Message-ID: 200412061400.iB6E0bFw005975@relay2.nnco.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Executive summary: We just did a cutover from a RedHat 8.0 box to a RedHat
Enterprise Linux 3 box and we're seeing a lot more swapping on the new box
than we ever did on the old box ... this is killing performance ...

Background:

Old Box:
RedHat 8.0
2GB Memory
Dual PIII 600MHz
Postgres 7.3.4
SHMMAX = 1073741824 (1 GB)
shared_buffers = 65536 (roughly 0.5 GB)
max_fsm_relations = 1000
max_fsm_pages = 1000000
vacuum_mem = 131072
Roughly 25 - 30 connections open (mostly idle) at any given time
(connection pools)

New Box:
RedHat Enterprise Linux ES 3
2GB Memory
Dual P4 Xeon 2.7 GHz
Postgres 7.3.4
SHMMAX = 1610612736 (1.5 GB)
shared_buffers = 131072 (roughly 1GB)
max_fsm_relations = 10000
max_fsm_pages = 10000000
sort_mem = 4096
vacuum_mem = 262144
Roughly 25 - 30 connections open (mostly idle) at any given time
(connection pools)

Both boxes are dedicated DB servers ... With the new configuration, we were
seeing swap rates of 1000-5000 KB/s (according to vmstat) ... with the old
configuration, we never saw any swapping ... I turned the shared_buffers and
sort_mem down on the new box to match the settings of the old box and found
that it reduced the swapping significantly (roughly 0-2000 KB/s) but didn't
eliminate it completely ... when looking at 'top' on the new box, the list
of postgres processes all seem to be indicating a different amount of memory
usage ... under the periods of heavy swapping, one or more of the postgres
processes would be way up there (between 500MB and 1000MB (which would
easily explain the swapping)) ... the question is: why aren't all of the
processes sharing the same pool of shared memory since I thought that's what
I'm doing when adjusting the shared_buffers property?

Here's an example of my 'top' (not under heavy load) demonstrating the
different memory usage by each postgres process ... I unfortunately don't
have the same snapshot data from the old configuration, but I seem to recall
that all of the postgres processes had

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
16966 postgres 15 0 107M 107M 105M S 0.0 5.3 0:39 1 postmaster
20198 postgres 15 0 40448 39M 37752 S 0.2 1.9 0:07 0 postmaster
18801 postgres 15 0 21932 21M 19616 S 0.0 1.0 0:01 0 postmaster
19210 postgres 16 0 21276 20M 19008 S 0.0 1.0 0:01 0 postmaster
19507 postgres 15 0 15504 14M 13580 S 0.0 0.7 0:00 3 postmaster
20308 postgres 15 0 12832 12M 11248 S 0.0 0.6 0:00 3 postmaster
20456 postgres 15 0 12500 12M 10920 S 0.0 0.6 0:00 1 postmaster
20403 postgres 15 0 11572 11M 9928 S 0.2 0.5 0:00 1 postmaster
20251 postgres 15 0 10796 10M 9260 S 0.0 0.5 0:00 0 postmaster
20398 postgres 15 0 10792 10M 9256 S 0.0 0.5 0:00 2 postmaster
20306 postgres 21 0 9100 8808 7796 S 0.0 0.4 0:00 1 postmaster
20425 postgres 16 0 9100 8808 7796 S 0.0 0.4 0:00 0 postmaster
20360 postgres 15 0 9096 8804 7792 S 0.0 0.4 0:00 3 postmaster
20383 postgres 21 0 9096 8804 7792 S 0.0 0.4 0:00 0 postmaster
20434 postgres 21 0 9096 8804 7792 S 0.0 0.4 0:00 1 postmaster
20305 postgres 15 0 9108 8796 7804 S 0.0 0.4 0:00 2 postmaster

Can anyone think of a reason as to why I'm seeing such heavy swapping?
According to Bruce Momjian's performance tuning guide, he recommends roughly
half the amount of physical RAM for the shared_buffers ... I tried turning
UP the shared_buffers even higher (to 180,000 i believe; roughly 1.5GB) and
that seemed to make the problem even worse ...

Thanks in advance,
Dave

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Daniel Martini 2004-12-06 14:46:12 Re: When to encrypt
Previous Message Richard Huxton 2004-12-06 14:01:53 Re: Delete function