Data Warehousing Tuning

From: "Paul Johnson" <paul(at)oxton(dot)com>
To: pgsql-performance(at)postgresql(dot)org
Subject: Data Warehousing Tuning
Date: 2005-07-06 17:15:47
Message-ID: 3688.217.45.209.171.1120670147.squirrel@www.gradwell.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

Hi all, we have the following setup:

- Sun V250 server
- 2*1.3GHz Sparc IIIi CPU
- 8GB RAM
- 8*73GB SCSI drives
- Solaris 10
- Postgres 8

Disks 0 and 1 are mirrored and contain the OS and the various software
packages, disks 2-7 are configured as a 320GB concatenation mounted on
/data, which is where load files and Postgres database and log files live.

The box is used by a small number of developers doing solely
Postgres-based data warehousing work. There are no end-users on the box,
and we are aiming for the maximum IO throughput.

Questions are as follows:

1) Should we have set the page size to 32MB when we compiled Postgres?

We mainly do bulk loads using 'copy', full-table scans and large joins so
this would seem sensible. Tables are typically 10 million rows at present.

2) What are the obvious changes to make to postgresql.conf?

Things like shared_buffers, work_mem, maintenance_work_mem and
checkpoint_segments seem like good candidates given our data warehousing
workloads.

3) Ditto /etc/system?

4) We moved the pg_xlog files off /data/postgres (disks 2-7) and into
/opt/pg_xlog (disks 0-1), but it seemed like performance decreased, so we
moved them back again.

Has anyone experienced real performance gains by moving the pg_xlog files?

Thanks in anticipation,

Paul.

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Alvaro Nunes Melo 2005-07-06 17:58:31 Storing data and indexes in different disks
Previous Message Tom Lane 2005-07-06 15:06:02 Re: Heavy virtual memory usage on production system