Re: Merged Model for libpq

On 2 April 2011 03:47, John R Pierce <pierce(at)hogranch(dot)com> wrote:

> how would you implement SQL without parsing, etc?    Annamali asked specifically for an implementation of the existing client-server protocol without TCP/IP, and thats exactly what the Unix socket interface is.
>

Maybe a little background here would help to understand our situation.
We have an in-memory storage engine implemented inhouse and we have
successfully ported postgresql engine (the sql engine) on top of our
in-memory storage engine. So what we have is:

Postgres SQL Engine + Our proprietary main-memory storage engine

We have introduced our own C API to write clients. For client-server
(CS) model, we have this C API implemented on top of libpq library.
For embedded model (EM) (client and server in same process, as
different threads), we have this C API implemented using SPI. So we
have two libraries of this C API, one is CS model and the other is EM
model. A client that uses this C API can be either linked to CS
library or EM library based on their needs. The application program
itself need not be modified.

Now, we are trying to see whether we can avoid this C API layer and
instead implement the libpq itself using the SPI interface. If we do
this, then any libpq client can either be client-server or embedded.
In this context, I am trying to explore whether for the embedded model
of libpq, using the SPI interface is the only option. Or would you
recommend using some other approach for client and server
communication when they run in the same process?

So Unix domain sockets is not satisfactory for us. We need something
with better performance because client and server are in same process
and in different threads. In this context, I was trying to find out
if the client-server protocol can be implemented without involving
sockets. Since the client and server are in same process (diff
threads), would it be possible to implement the protocol using
something like ACE_Message_Queue? If we do this then serialization of
objects would not be necessary and lot of data copy can be avoided.
We can just pass pointers from server to client (wherever
appropriate). I thought that this can be an alternative to using the
SPI. But is this feasible? Is the client-server protocol, as
implemented now, amenable to such refactoring?

It is a big story, but I thought the background will help highlight
our context. Can you guys provide more information that would help us
to make informed decisions?

Thank you.

Rgds,
anna

--
'உண்மை ஒருநாள் வெளியாகும், அதில் உள்ளங்களெல்லாம் தெளிவாகும்.
பொறுமை ஒருநாள் புலியாகும், அதற்கு பொய்யும் புரட்டும் பலியாகும்.'
                                     - பட்டுக்கோட்டை கல்யாணசுந்தரம்
http://www.youtube.com/watch?v=0J71uLUvjnU&feature=related