The overall experience of TPC-V benchmark team with PostgreSQL

Hello PGSQL fans,
Looking back at my posts the past couple of days and the replies that I've got, I realized that I have failed to make one point clear: we are very pleased with what we have seen from PostgreSQL so far. Let me explain. At this point of developing or porting a benchmark on a new DBMS, the team usually deals with stability, scalability, or fundamental performance issues. Our fear was that working with an open source DBMS, we'd experience more issues than usual. But we got the kit running transactions on PGSQL quickly, and after some early tests, I decided to try the kit on a larger testbed (two other folks are the developers of the benchmark code; I design, run, and analyze the experiments). I have the benchmark running on a 300,000-customer database on a 16-CPU system, unusual for this early in the prototyping phase. People who developed TPC-E (the father of our benchmark) did their prototyping on commercial databases with much smaller databases on smaller systems. On this large testbed, PGSQL has been working like a champ, and performance is what I would call decent. Put in other words, I have been pleasantly surprised by the throughput I am getting out of the system, saturating a 16-way with no visible signs of contention when we reduce the database size.

We are developing a "reference" kit. People are not obligated to use it to publish official results. They can use it to kick the tires, then go to one of the commercial DBMS vendors and ask for their kit for an official TPC-V publication. Even if that's all that people do with the reference kit, our team has achieved the goal that the TPC set for us. What I am trying to do is see if we can take this to the point that people use PGSQL to publish official results and use it in competitive situations. It looks possible, so I'd love to see it happen.

Again, overall, our experience with PGSQL has been positive, even in terms of performance.

Thanks,
Reza