Re: Shared memory

Martijn,

I tried a Socket approach. Using the new IO stuff that arrived with Java 1.4 (SocketChannel
etc.), the performance is really good. Especially on Linux where an SMP machine show a 1 to
1.5 ratio between one process doing ping-pong between two threads and two processes doing
ping-pong using a socket. That's acceptable overhead indeed and I don't think I'll be able
to trim it much using a shared memory approach (the thread scenario uses Java monitor locks.
That's the most efficient lightweight locking implementation I've come across).

One downside is that on a Windows box, the ratio between the threads and the processes
scenario seems to be 1 to 5 which is a bit worse. I've heard that Solaris too is less
efficient then Linux in this respect.

The real downside is that a call from SQL to PL/Java using the current in-process approach
is really fast. It takes about 5 micro secs on my 2.8GHz i386 box. The overhead of an
IPC-call on that box is about 18 micro secs on Linux and 64 micro secs on Windows. That's an
overhead of between 440% and 1300% due to context switching alone. Yet, for some
applications, perhaps that overhead is acceptable? It should be compared to the high memory
consumption that the in-process approach undoubtedly results in (which in turn might lead to
less optimal use of CPU caches and, if memory is insufficient, more time spent doing swapping).

Given those numbers, it would be interesting to hear what the community as a whole thinks
about this.

Kind Regards,
Thomas Hallgren

Martijn van Oosterhout wrote:
> On Fri, Mar 24, 2006 at 11:51:30AM +0100, Thomas Hallgren wrote:
>> Hi,
>> I'm currently investigating the feasibility of an alternative PL/Java
>> implementation that would use shared memory to communicate between a JVM
>> and the backend processes. I would very much like to make use of the
>> routines provided in shmem.c but I'm a bit uncertain how to add a segment
>> for my own use.
>
> I'm wondering if a better way to do it would be similar to the way X
> does it. The client connects to the X server via a pipe (tcp/ip or unix
> domain). This is handy because you can block on a pipe. The client then
> allocates a shared memory segment and sends a message to the server,
> who can then also connect to it.
>
> The neat thing about this is that the client can put data in the shared
> memory segment and send one byte through the pipe and then block on a
> read. The JVM which has a thread waiting on the other end wakes up,
> processes the data, puts the result back and writes a byte to the pipe
> and waits. This wakes up the client who can then read the result.
>
> No locking, no semaphores, the standard UNIX semantics on pipes and
> sockets make sure everything works.
>
> In practice you'd probably end up sending small responses exclusively
> via the pipe and only use the shared memory for larger blocks of data
> but that's your choice. In X this is mostly used for image data and
> such.
>
>> My questions are:
>> 1. Do you see something right away that invalidates this approach?
>
> Nothing direct, though a single segment just for finding the JVM seems
> a lot. A socket approach would work better I think.
>
>> 2. Is using the shared memory functionality that the backend provides a
>> good idea (I'm thinking shmem functions, critical sections, semaphores,
>> etc.). I'd rather depend on them then having conditional code for different
>> operating systems.
>
> That I don't know. However, ISTM a lock-free approach is better
> wherever possible. If you can avoid the semaphores altogether...
>
>> 3. Would it be better if the Postmaster allocated the global segment and
>> started the JVM (based on some config parameter)?
>
> I don't know about the segment but the postmaster should start. I
> thought the tsearch guys had an approach using a co-process. I don't
> know how they start it up but they connected via pipes.
>
> Hope this helps,