Re: Is there a meaningful benchmark?

You have a point, as do a lot of the other folks.

However, keeping the KISS principle in mind, you can create a benchmark
that simply sets up a sample database and forks off a bunch of processes
to do random updates for an hour, say. Dead simple.

In fact, it's so simple that I've already written the code and have it
running against Postgres now. A Perl DBI script runs in a loop
updating, and later prints out the number of transactions it completed
in the given time frame. At the end I just tally up the numbers and I
have the Will Rutherdale benchmark number for Postgres. It will give me
a simple number in units of transactions per second.

When I get time I'll set up an equivalent MySQL database on the same
machine and run it against that.

I have 'top' running in one screen and can see all the copies of my
script and all the copies of postmaster and the CPU they're using.

The degree to which I got excoriated just for asking the question
convinced me that I was onto something and had to do it myself.

-Will

-----Original Message-----
From: Scott Ribe [mailto:scott_ribe(at)killerbytes(dot)com]
Sent: 20 March 2009 11:27
To: Will Rutherdale (rutherw); pgsql-general(at)postgresql(dot)org
Subject: Re: [GENERAL] Is there a meaningful benchmark?

> That's why I was looking for a more balanced benchmark that exercises
> said capabilities.

OK, here's the thing, I will give you *one* sample issue to think about,
as
an illustration of the kinds of differences there are.

- PostgresQL uses MVCC instead of row or page locking, which means only
writer vs writer locks, which means in many situations less contention
and
better throughput as your concurrent load goes up.

- But it also means slower count(*) and no covering indexes. (Keeping
multiple versions of index pages would be too slow for many other
operations, so PG has go to the table pages to check which rows are
actually
visible in the current transaction.)

See? Overall, PG performs very well and is certainly comparable to the
"big
boys" on the same hardware. But the strengths & weaknesses can be *very*
specific to particular queries.

--
Scott Ribe
scott_ribe(at)killerbytes(dot)com
http://www.killerbytes.com/
(303) 722-0567 voice