Re: Postgres for a "data warehouse", 5-10 TB

2011/9/11 pasman pasmański <pasman(dot)p(at)gmail(dot)com>:
> For 10 TB table and 3hours, disks should have a transfer about 1GB/s (seqscan).

Random data point. Our primary servers were built for OLTP with 48
cores and 32 15kSAS drives. We started out on Arecas but the
Supermicro 1Us we were using didn't provide enough cooling and the
Arecas were burning out after 2 to 4 months, so on those machines, we
pulled the Arecas and replaced them with simple LSI SAS non-RAID
cards. Both were RAID-10, the latter with linux software RAID.

With the Arecas the OLTP performance is outstanding, garnering us
~8500tps at 40 to 50 threads. However, sequentual performance was
just so so at around read / write speeds of 500/350MB/s. The SW
RAID-10 can read AND write at right around 1GB/s. what it lacks in
transactional throughput it more than makes up for in sequential read
/ write performance.

Another data point. We had a big Oracle installation at my last job,
and OLAP queries were killing it midday, so I built a simple
replication system to grab rows from the big iron Oracle SUN box and
shove into a single core P IV 2.xGHz machine with 4 120G SATA drives
in SW RAID-10.

That machine handily beat the big iron Oracle machine at OLAP queries,
running in 20 minutes what was taking well over an hour for the big
Oracle machine to do, even during its (Oracle machine) off peak load
times.