Postgresql performance: GCC 2 vs GCC 3.1 on MacOS X.II

Hi folks,

I ran several iterations of the postgresql 7.2.1 "runwisc.sh"
benchmark on an 800Mhz Apple TiBook with 512Mb RAM. The first run was
compiled using MacOS X.I, the second set was the same executable on
MacOS X.II, the third was a recompile using the GCC 3.1 based compiler
shipped by apple (Build 1151), and the fourth was another GCC 3.1
recompile, but this time with -faltivec .

Version Mean clock time
MacOS X.I w/ gcc2 : 22.06 seconds
MacOS X.II w/ gcc2 : 13.52 seconds
MacOS X.II w/ gcc3 : 13.16 seconds
MacOS X.II w/ gcc3 -faltivec : 13.31 seconds

I'll let the difference between MacOS X.I and "Jaguar" pass -- I only
ran the benchmark once on X.I, and it's possible that the big
difference could have come from the UI or some other factor.

A T-test reports that the difference between the average with gcc2 and
gcc3 is significant. However, the difference is not all very high.
This could be because the bench was run on a laptop, and thus may have
been I/O bound. The difference between gcc without attempting to use
altivec and gcc with altivec was on the borderline of significance,
with -faltivec actually slowing the system down!

Below: statistics, methods, raw data, bad things about how I generated
the numbers.

Statistics

T-Test: MacOS X.II w/ gcc2 vs. MacOS X.II w/gcc3

Variable Method Variances DF t Value Pr > |t|

benchtime Pooled Equal 8 3.75 0.0056
benchtime Satterthwaite Unequal 3.48 3.12 0.0428

T-Test: MacOS X.II w/ gcc3 vs. MacOS X.II w/gcc3 & -faltivec

Variable Method Variances DF t Value Pr > |t|

benchtime Pooled Equal 10 -1.83 0.0971
benchtime Satterthwaite Unequal 6.55 -1.83 0.1128

Methodology

make; make install; to produce the postgresql executable

To run the benchmark
cd test/bench
./create.sh template1
time ./runwisc.sh template1
Through out the first result, and rerun.

Raw data

Original (OS X.I)
0:22.06

Round II (OS X.II, same version as above)
0:13.51
0:13.54
0:13.24
0:13.79

Round III (OS X.II w/ gcc 3.1)
Apple Computer, Inc. GCC version 1151, based on gcc version 3.1
20020420 (prerel
ease)
0:13.09
0:13.18
0:13.28
0:13.12
0:13.07
0:13.19

Round IV (OS X.II w/ gcc 3.1 + -faltivec)
0:13.40
0:13.30
0:13.12
0:13.12
0:13.64
0:13.29

Problems

The UI was running when I generated the data; it might have removed
uncertainty if I'd run the benchmarks while ssh'd in. This may have
accounted for some of the huge gap between MacOS X.I and "Jaguar"
performance.

Although I can't really say that the gap definitely existed, because I
only have data from one run of the MacOS X.I benchmark

And, of course, more observations elsewhere would have come in handy,
especially to resolve the GCC 3 with and without altivec difference.

To help remove I/O from consideration, the benchmark database could
have been placed on a ramdisk.

Yours truly,
Jeffrey Boulier

--
If you'd like to send me a personal e-mail, please remove all but the
search engine from the front part of my e-mail address. Spam not
welcome.
Psalm 34:13 "Keep your tongue from evil, and your lips from speaking
guile"