Remote PL/Java, Summary

Hi all,
And thanks for very good input regarding a remote alternative to PL/Java
(thread titled "Shared Memory"). I'm convinced that such an alternative
would be a great addition to PL/Java and increase the number of users.
The work to create such a platform that has the stability and quality of
todays PL/Java is significant (I really do think it is a
production-grade product today). So significant in fact, that I'm
beginning to think of a third alternative. An alternative that would
combine the performance of using in-process calls with the benefits of
sharing a JVM. The answer is of course to make the backend multi-threaded.

This question has been debated before and always promptly rejected. One
major reason is of course that it will not bring any benefits over the
current multi-process approach on a majority of the platforms where
PostgreSQL is used. A process-switch is just as fast as a thread-switch
on Linux based systems. Over the last year however, something has happen
that certainly speaks in the favor of multi-threading. PostgreSQL is
getting widely adopted on Windows. On Windows, a process-switch is at
least 5 times more expensive then a thread-switch. In order to
appropriate locking, PostgreSQL is forced to do a fair amount of
switching during transaction processing so the gain in using a
multi-threaded approach on Windows is probably significant. The same is
true for other OS'es where process-switching is relatively expensive.

There are other benefits as well. PostgreSQL would no longer need shared
memory and semaphores and lot more resources could be shared between
backend processes. The one major drawback of a multi-threaded approach
(the one that's been the main argument for the defenders of the current
approach) is vulnerability. If one thread is messing things up, then the
whole system will be brought to a halt (on the other hand, that can be
said about the current shared-memory approach as well). The cure for
this is to have a system that, to the extent possible, prevents this
from happening. How would that be possible? Well, such systems are
widely used today. Huge companies use them in mission critical
applications all over the world. They are called Virtual Machines. Two
types in particular are gaining more an more ground. The .NET based CLR
and the Java VM.

Although there's an Open Source initiative called Mono that implements
the CLR, I still don't see it as a viable alternative to create a
production-grade multi-platform database. Microsofts CLR is of course
confined to Microsoft platforms. The Java VM's are however a different
matter altogether. And with the java.nio.channels package that was
introduced in Java 1.4 and the java.util.concurrent package from Java
5.0, Java has taken a major steps forward in being a very feasible
platform for a database implementation. There's actually nothing
stopping you from doing a high-performance MVCC system using Java today.
A SQL parser would be based on JavaCC technology (the grammar is already
written although it needs small adjustments to comply with the
PostgreSQL dialect). Lots of technology is there out-of-the-box such as
regular expressions, hash-maps, linked lists, etc. Not to forget an
exceptionally great threading system, now providing atomic operations,
semaphores, copy-on-write arrays etc. In short, everything that a
database implementor could ever wish for.

The third alternative for PL/Java, an approach that gets more viable
every minute I think about it, is to implement the PostgreSQL backend
completely in Java. I'm involved in the development of one of the
commercial JVM's. I know that an enormous amount of resources are
constantly devoted to performance optimizations. The days when a complex
system written in C or C++ could outperform a JVM have passed. A static
optimizer can only do so well. A JVM, that collects heuristics,
communicates with the CPU about cache usage etc., can be a great deal
smarter on how the final machine code will be optimized, and
re-optimized should the conditions change. It would be great if
PostgreSQL could benefit from all this research.

If a commercial JVM is perceived as a problem, then combine^h^h^hpile
the code with GNU gcj instead of gcc like today.

The list of advantages can be made a mile long. There's no point in
listing everything here. From my own standpoint, I'm of course thinking
first and foremost about the advantages with PL/Java. It will become the
absolute most efficient PL of them all. Other languages, for which no
good Java implementation exists (I'm thinking Jython for Python, etc.),
can be implemented using JNI. The most common functions used by say,
PL/Perl could probably be implemented as callbacks into the Java domain
in order to make the changes in the respective PL minimal.

Opinions? Suggestions?

Kind Regards,
Thomas Hallgren