Re: [pgsql-advocacy] Benchmarks WAS: Sun Talks about MySQL

"Josh Berkus" <josh(at)agliodbs(dot)com> writes:

>> I think TPC-E will make both of these major improvements much more important.
>> I suspect it would be hard to get 8.2 to even pass TPC-E due to the checkpoint
>> dropouts.
>
> You'd be surprised, then. We're still horribly, horribly lock-bound on TPC-E;
> on anything over 4 cores lock resolution chokes us to death. See Jignesh's and
> Paul's various posts about attempts to fix this.

Most of those posts have been about scalability issues with extremely large
numbers of sessions. Those are interesting too and they may be limiting our
results in benchmarks which depend on such a configuration (which I don't
think includes TPC-E, but the benchmark Jignesh has been writing about is some
Java application benchmark which may be such a beast) but they don't directly
relate to whether we're "passing" TPC-E.

What I was referring to by "passing" TPC-E was the criteria for a conformant
benchmark run. TPC-C has iirc, only two relevant criteria: "95th percentile
response time < 5s" and "average response time < 95th percentile response
time". You can pass those even if 1 transaction in 20 takes 10-20s which is
more than enough to cover checkpoints and other random sources of inconsistent
performance.

TPC-E has more stringent requirements which explicitly require very consistent
response times and I doubt 8.2 would have been able to pass them. So the
performance limiting factors whether they be i/o, cpu, lock contention, or
whatever don't even come into play. We wouldn't have any conformant results
whatsoever, not even low values limited by contention. 8.3 however should be
in a better position to pass.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's 24x7 Postgres support!