Re: Benchmark comparing PostgreSQL, MySQL and Oracle

Hi all,

As the author of BenchmarkSQL and the founder of EnterpriseDB.... I
can assure you that BenchmarkSQL was NOT written specifically for
PostgreSQL. It is intended to be a completely database agnostic
tpc-c like java based benchmark.

However; as Jonah correctly points out in painstaking detail:
PostgreSQL is good, but... Oracle, MySQL/Innodb and and and don't
necessarily suck. :-)

--Luss

PS: Submit a patch to BenchmarkSQL and I'll update it.

On 2/20/09, Sergio Lopez <sergio(dot)lopez(at)nologin(dot)es> wrote:
> El Fri, 20 Feb 2009 16:54:58 -0500
> Robert Haas <robertmhaas(at)gmail(dot)com> escribió:
>
>> On Fri, Feb 20, 2009 at 4:34 PM, Jonah H. Harris
>> <jonah(dot)harris(at)gmail(dot)com> wrote:
>> > On Fri, Feb 20, 2009 at 3:40 PM, Merlin Moncure
>> > <mmoncure(at)gmail(dot)com> wrote:
>> >>
>> >> ISTM you are the one throwing out unsubstantiated assertions
>> >> without data to back it up. OP ran benchmark. showed
>> >> hardware/configs, and demonstrated result. He was careful to
>> >> hedge expectations and gave rationale for his analysis methods.
>> >
>> > As I pointed out in my last email, he makes claims about PG being
>> > faster than Oracle and MySQL based on his results. I've already
>> > pointed out significant tuning considerations, for both Postgres
>> > and Oracle, which his benchmark did not take into account.
>> >
>> > This group really surprises me sometimes. For such a smart group
>> > of people, I'm not sure why everyone seems to have a problem
>> > pointing out design flaws, etc. in -hackers, yet when we want to
>> > look good, we'll overlook blatant flaws where benchmarks are
>> > concerned.
>>
>> The biggest flaw in the benchmark by far has got to be that it was
>> done with a ramdisk, so it's really only measuring CPU consumption.
>> Measuring CPU consumption is interesting, but it doesn't have a lot to
>> do with throughput in real-life situations. The benchmark was
>> obviously constructed to make PG look good, since the OP even mentions
>> on the page that the reason he went to ramdisk was that all of the
>> databases, *but particularly PG*, had trouble handling all those
>> little writes. (I wonder how much it would help to fiddle with the
>> synchronous_commit settings. How do MySQL and Oracle alleviate this
>> problem and we can usefully imitate any of it?)
>>
>
> The benchmark is NOT constructed to make PostgreSQL look good, that
> never was my intention. All databases suffered the I/O bottleneck for
> their redo/xlog/binary_log files, specially PostgreSQL but closely
> followed by Oracle. For some reason MySQL seems to deal better with I/O
> contention, but still gives numbers that are less than the half it gives
> with tmpfs.
>
> While using the old array (StorageTek T3), I've played with
> synchronous_commit, wal_sync_method, commit_delay... and only setting
> wal_sync_method = open_datasync (which, in Solaris, completly disables
> I/O syncing) gave better results, for obvious reasons.
>
> Anyway, I think that in the next few months I'll be able to repeat the
> tests with a nice SAN, and then we'll have new numbers that will be
> more near to "real-world situations" (but synthetic benchmarks always
> are synthetic benchmarks) and also we'll be able to compare them with
> this ones to see how I/O contetion impacts on each database.
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>