Re: internet week article

Eric Kolve wrote:
> http://www.internetweek.com/reviews01/rev090301.htm
>
> I would be most interested to here this lists response to the following
> paragraph. Though much of the specifics of the benchmark are left out,
> so tuning could very much of been an issue.
>
> Database performance is a key issue for most businesses.
> If the database is slow, the business is too. Our benchmarks
> showed that PostgreSQL is slower than Oracle 8.1.7 in many
> operations. The tests were performed on a 1-GHz Pentium III
> system with 1 GB of RAM running Red Hat 7.1.
>
> Oracle performed the count 33 percent faster and the distinct
> count 27 percent faster. The simple query was approximately
> 18 percent faster in Oracle, but the most concerning was the
> slowdown of PostgreSQL in the two- and three-table join test.

We are aware of the fact, that MVCC and not beeing able to
shortcut aggregates have an impact on things like count().
Would of course be interesting to get more details (full
schema, queries and if DB was vacuumed properly).

OTOH this kinda stuff is IMHO totally braindead. All
benchmarks from the Transaction Processing Council (TPC)
define a system under test (SUT) and the values you get from
them are maximum troughput and cost per transaction for the
given implementation/setup. And there are "really good"
reasons behind it.

1. Why defining a SUT?

The raw database speed itself on two or three different
queries is meaningless. Nobody uses SQL as the user
interface in the real world. The entire business logic is
allways implemented in an application AND the database
together, with amounts varying (some TPC specs allow to
implement a given feature of the SUT either in the
application or on the DB side as stored procedure etc.).

You're interested in how fast the user gets the response
for his form. The functionality it provides can be
implemented in a couple of different ways, depending on
the features provided by the middleware and/or the
database, why a simple comparision of "how fast is
count(*)" is nonsense.

2. When to look at maximum throughput

If you are looking for speed, only speed and nothing but
speed, you want to know how high you can scale your
system. The cost in that case is not really relevant, so
you're happy if you find a vendor selling you hardware
for a couple million bucks and a DB vendor you can pay
another million or two every year, as long as these
products get the job done in time (budget doesn't
matter).

You want to know "does this combo handle 5,000
simultaneous users with 7.5 terabytes of data". If it
doesn't, it is irrelevant if you can afford it, it
doesn't fit your needs anyway.

3. When to look at cost per transaction

If you want to get your job done for a reasonable price,
you're looking for a good cost per transaction. In the
TPC specs, the full disclosure report has to include OS,
DB and Middleware license fees as well as the hardware
and maintainence costs, plus average administration
expenses and support contracts. So it's an estimation of
the total cost to run THIS setup in production.

You want to know if you can afford the installation. The
question is "does it get my job done in time and
budget?".

4. Why the truth is in between

If you visit www.tpc.org, you can find a collection of
presentation slides which include a two dimensional chart
with dot's of full disclosure reports for the TPC/C
benchmark. These dot's mostly show that MS/SQL server is
usually cheaper than Oracle on various platforms, while
Oracle dominates the high-end troughput area (no dot's at
all for MS there - so that OS/DB simply sucks if you're
interested in speed only).

You want to know "does it get my job done in time and
budget NOW?". But you also want to know if it'll get the
job done in the future, when your business increased as
well as your budget.

At some point in the future, your business might outgrow the
capabilities of your today choice. At that time, you'll have
to migrate (back) to the high-end solution and that's why I
(personally) accept Oracle compatibility ugly-kludges over
SQL92 evangelism sometimes.

The bottom line is, that if you just throw half of the Oracle
license fees into a hardware upgrade, I'm sure the
"benchmark" you're talking about will show totally different
numbers, right? So what does that "benchmark" really tell?
For me it tells "if you spend more money you get better
performance". Well, that is a general rule of thumb we all
knew already, tell me something new.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com