Re: Other benchmark than OSDB

On Tue, 17 Sep 2002 12:30:25 -0700, Andrew Sullivan wrote:

> On Tue, Sep 17, 2002 at 11:53:05AM -0700, Derek Neighbors wrote:
>> Someone needs to check out
>> http://www.osdl.org/projects/performance/osdldbt.html the OSDL
>> benchmarking tool for DBs and get Postgres to work with it and then
>> run it and publish results or at a mininum use it to help tune
>> performance for Postgres.
>
> I'm afraid that the OSDL benchmark has a (IMNSHO stupid) handicap in
> getting it working and making it a well-respected benchmark:

There is no handicap, it just needs to be done. Stored procedures need to
be written for postgres. You can't always expect stored procedures for
one database to work on another. Similarly, while a lot of databases
support ODBC, you can't expect ODBC to be used for every database. We
are very willing to support and aid in getting postgres to work with our
test kits, but we do not have the resources to commit someone here to do
all the work. Please contact me (markw(at)osdl(dot)org) if anyone wishes to help.

> "While the inspiration for OSDL-DBT-1 is the TPC-W, they are entirely
> different workloads and results obtained should not and cannot be
> compared. Commercial use of results obtained by running OSDL-DBT-1
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> are expressly prohibited."
> ^^^^^^^^^^^^^^^^^^^^^^^^
>
> So, it's not TPC, you can't compare it to TPC, and if you wanted to
> compete with the TPC using this test, you couldn't do it. What's worse,
> since there's no definition of "commercial use" that I was able to find,
> I'm not sure if it extends to such things as (for instance) writing case
> studies or responses to RFPs. Even stranger is that the Sourceforge
> page says the project is released under the Artistic License, which sure
> doesn't include restrictions on commercial use. (Indeed, the point of
> the open source brand is supposedly that it makes free software more
> appealing to business. Oh, well. :-/ )

"Commercial use" can be generalized to mean that you cannot use our test
kits to promote sales for your product. I hope it is understood that
there are legal ramifications regarding comparing results from our test
kits to TPC results.

I do not follow the jump from restrictions for commercial use to the writing
of case studies. For example, there are no issues for using our test kits to
write a case study that surveys the performance of various open-source
databases. Now you cannot publish performance results for a lot of
commercial databases because they have clauses against doing such
publications attached to the license of their use. I hope none of the
open-source databae are licensed this way.

> I'm sure my boss would _love_ for me to work on something we can't ever
> use. Sheesh.

What exactly do you want to do it for? I can certainly verify if it's
possible or not.

> A

-----
Mark Wong - - markw(at)osdl(dot)org
Open Source Development Lab Inc - A non-profit corporation
15275 SW Koll Parkway - Suite H - Beaverton OR, 97006
(503)-626-2455 x 32 (office)
(503)-626-2436 (fax)
http://www.osdl.org/archive/markw/