Re: Benchmark comparing PostgreSQL, MySQL and Oracle

On Fri, Feb 20, 2009 at 6:28 AM, Sergio Lopez <sergio(dot)lopez(at)nologin(dot)es>wrote:

> Hi,
>
> I've made a benchmark comparing PostgreSQL, MySQL and Oracle under three
> environments: GNU/Linux-x86, Solaris-x86 (same machine as GNU/Linux) and
> Solaris-SPARC. I think you might find it interesting:
>
>
> http://blogs.nologin.es/slopez/archives/17-Benchmarking-Databases-I.-Volatile-Storage..html

Sorry Segio,

In addition to violating your Oracle license, you need to learn a couple
things about benchmarking.

First of all, you need to do some research on the benchmark kit itself,
rather than blindly downloading and using one. BenchmarkSQL has significant
bugs in it which affect the result. I can say that authoritatively as I
worked on/with it for quite awhile. Don't trust any result that comes from
BenchmarkSQL. If you fix the bugs, Oracle (out of the box in OLTP config)
will come out 60%.

Oracle comes out twice as fast as PG on Linux. And, unless you're using a
significant number of warehouses, MySQL+InnoDB will come out better than PG
as well.

Second, I didn't see anything in your Oracle settings for parallelism and
I/O tuning. Did you set them? And, based on what you presented, you didn't
set configure the SGA appropriately given the hardware mentioned. What was
your log buffer set to?

Third, did you manually analyze the Oracle/MySQL databases, because
BenchmarkSQL will automatically analyze Postgres' tables to help the
optimizer... did you do the same for the other databases?

Fourth, it didn't look like you tuned PG properly either. What was
shared_buffers, wal_buffers, and wal_sync_method set to?

Fifth, did you do an out-of-the-box install of Oracle, or a custom one? If
out of the box, did you choose OLTP or General?

There's lots of other things I could go on about in regard to flushing all
the caches prior to starting the benchmarks, filesystem options, etc.

Not trying to be rude, but *THIS* is why Oracle, IBM, Microsoft, et al.
don't want people running benchmarks without their permission. When
performing benchmarks, there are a lot of things to take into
consideration. If you're just performing out-of-the-box tests, then that's
fine, but you have to make sure the benchmark kit doesn't optimize itself
for any one of those databases (which it does for PG).

--
Jonah H. Harris, Senior DBA
myYearbook.com