Re: Hardware recommendations

John,

The platform is a network monitoring system, so we have quite a lot of inserts/updates (every data point has at least one record insert as well as at least 3 record updates). The management GUI has a lot of selects. We are refactoring the database to some degree to aid in the performance, since the performance degradations are correlated to the number of users viewing the system GUI.
My biggest concern with SSD drives is their life expectancy, as well as our need for relatively high capacity. From a purely scalability perspective, this setup will need to support terabytes of data. I suppose I could use table spaces to use the most accessed data in SSD drives and the rest on regular drives.
As I stated, I am moving to RAID 10, and was just wondering if the logs should still be moved off to different spindles, or will leaving them on the RAID10 be fine and not affect performance.

> -----Original Message-----
> From: John W Strange [mailto:john(dot)w(dot)strange(at)jpmchase(dot)com]
> Sent: Wednesday, December 08, 2010 4:32 PM
> To: Benjamin Krajmalnik; pgsql-performance(at)postgresql(dot)org
> Subject: RE: Hardware recommendations
>
> Ben,
>
> It would help if you could tell us a bit more about the read/write mix
> and transaction requirements. *IF* you are heavy writes I would suggest
> moving off the RAID1 configuration to a RAID10 setup. I would highly
> suggest looking at SLC based solid state drives or if your budget has
> legs, look at fusionIO drives.
>
> We currently have several setups with two FusionIO Duo cards that
> produce > 2GB second reads, and over 1GB/sec writes. They are pricey
> but, long term cheaper for me than putting SAN in place that can meet
> that sort of performance.
>
> It all really depends on your workload:
>
> http://www.fusionio.com/products/iodrive/ - BEST in slot currently
> IMHO.
> http://www.intel.com/design/flash/nand/extreme/index.htm?wapkw=(X25-E)
> - not a bad alternative.
>
> There are other SSD controllers on the market but I have experience
> with both so I can recommend both as well.
>
> - John
>
>
>
> -----Original Message-----
> From: pgsql-performance-owner(at)postgresql(dot)org [mailto:pgsql-performance-
> owner(at)postgresql(dot)org] On Behalf Of Benjamin Krajmalnik
> Sent: Wednesday, December 08, 2010 5:04 PM
> To: pgsql-performance(at)postgresql(dot)org
> Subject: [PERFORM] Hardware recommendations
>
> I need to build a new high performance server to replace our current
> production database server.
> The current server is a SuperMicro 1U with 2 RAID-1 containers (one for
> data, one for log, SAS - data is 600GB, Logs 144GB), 16GB of RAM,
> running 2 quad core processors (E5405 @ 2GHz), Adaptec 5405 Controller
> with BBU.  I am already having serious I/O bottlenecks with iostat -x
> showing extended periods where the disk subsystem on the data partition
> (the one with all the random i/o) at over 85% busy.  The system is
> running FreeBSD 7.2 amd64 and PostgreSQL 8.4.4 on amd64-portbld-
> freebsd7.2, compiled by GCC cc (GCC) 4.2.1 20070719  [FreeBSD], 64-bit.
> Currently I have about 4GB of shared memory allocated to PostgreSQL.
> Database is currently about 80GB, with about 60GB being in partitioned
> tables which get rotated nightly to purge old data (sort of like a
> circular buffer of historic data).
>
> I was looking at one of the machines which Aberdeen has (the X438), and
> was planning  on something along the lines of 96GB RAM with 16 SAS
> drives (15K).  If I create a RAID 10 (stripe of mirrors), leaving 2 hot
> spares, should I still place the logs in a separate RAID-1 mirror, or
> can they be left on the same RAID-10 container?
> On the processor front, are there advantages to going to X series
> processors as opposed to the E series (especially since I am I/O
> bound)?  Is anyone running this type of hardware, specially on
> FreeBSD?  Any opinions, especially concerning the Areca controllers
> which they use?
>
> The new box would ideally be built with the latest released version of
> FreeBSD, PG 9.x.  Also, is anyone running the 8.x series of FreeBSD
> with PG 9 in a high throughput production environment?  I will be
> upgrading one of our test servers in one week to this same
> configuration to test out, but just wanted to make sure there aren't
> any caveats others have experienced, especially as it pertains with the
> autovacuum not launching worker processes which I have experienced.
>
> Best regards,
>
> Benjamin
>
> --
> Sent via pgsql-performance mailing list (pgsql-
> performance(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
> This communication is for informational purposes only. It is not
> intended as an offer or solicitation for the purchase or sale of
> any financial instrument or as an official confirmation of any
> transaction. All market prices, data and other information are not
> warranted as to completeness or accuracy and are subject to change
> without notice. Any comments or statements made herein do not
> necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
> and affiliates.
>
> This transmission may contain information that is privileged,
> confidential, legally privileged, and/or exempt from disclosure
> under applicable law. If you are not the intended recipient, you
> are hereby notified that any disclosure, copying, distribution, or
> use of the information contained herein (including any reliance
> thereon) is STRICTLY PROHIBITED. Although this transmission and any
> attachments are believed to be free of any virus or other defect
> that might affect any computer system into which it is received and
> opened, it is the responsibility of the recipient to ensure that it
> is virus free and no responsibility is accepted by JPMorgan Chase &
> Co., its subsidiaries and affiliates, as applicable, for any loss
> or damage arising in any way from its use. If you received this
> transmission in error, please immediately contact the sender and
> destroy the material in its entirety, whether in electronic or hard
> copy format. Thank you.
>
> Please refer to http://www.jpmorgan.com/pages/disclosures for
> disclosures relating to European legal entities.