Re: strange postgresql x mysql comparison in forrester analyse

On Sun, 18 Oct 2009, Ron Mayer wrote:

> Would have been nice if they had pointed to the benchmark they had in
> mind. The only well known published benchmark I see (on spec.org) that
> compares postgres to many of these other databases made us look OK to
> me.

Found the talk I was alluding to:
http://blogs.sun.com/jkshah/entry/postgresql_east_2008_talk_postgresql
Note the TCP-H summary on P26. Out of the 21 queries in that standard
benchmark load, PostgreSQL basically doesn't handle 9 of them. Makes it
hard for businesses to trust you can deploy it as a generic database
application for data-warehouse purposes knowing there are some sizable
holes there. And it's difficult to push back and dispute claims of
benchmark issues with the database vs. the commercial products knowing
it's not hard to discover said holes.

> I think the problem is that we're comparing an apples (just the core
> postgres kernel) with oranges (the database kernels plus all the
> supporting tools and apps from other vendors).

Sure, but the larger point I was suggesting is that the way the analyst is
evaluating like that matches that of more business people than the
community may like to acknowledge. There's a reason people bundle all
that stuff with their core database products.

Ultimately this is a hard issue to field. If the comparison piece here
was instead EDB's bundling of Postgres with the whole fairly well
integrated development tool stack they make available now (note how big
the list at http://www.enterprisedb.com/products/download.do is getting
nowadays), the larger comparison might go off a bit better. You know that
would raise a completely different set of criticism of the study from
within this community though.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD