[Pljava-dev] UDT send and receive

Markus Schaber wrote:
> Hi, Thomas,
>
> Thomas Hallgren wrote:
>
>
>>> From looking at the code, it seems to assume that the send and receive
>>> code assumes that the internal (on-disk) representation is the same than
>>> the one used for binary I/O, rather than relying on the send and receive
>>> functions.
>>>
>> Not sure what you mean. The UDT functions *are* the send/receive
>> functions (with an added UDT parameter). They don't care much about the
>> representation as such. There's a length (-2, -1 or a verbatim length)
>> and then there are bytes of data. The send fucntion uses byteasend,
>> unknownsend, or a StringInfo depending on the length. The receive
>> performs the corresponding read. Totally representation agnostic.
>>
>
> Yes. And, as I said, they send the on-disk data 1:1.
>
> PostGIS internally uses a different storage format compare to binary
> input/output, for abstraction purposes, and the send and receive
> functions perform some conversions (including adding varlena header, and
> adopting byte order).
>
> In my eyes, your current approach will fail when someone gets a binary
> dump of the data via COPY, and then reloads the data on a platform with
> different endianness. And clients using binary V3 protocol need to know
> the server's endianness.
>
>
We're talking past each other here. The send/receive functions are the
ones that perform the conversion. If you do it in Java, you write
send/receive functions in Java. What you see is just the middle man,
passing data to/from such functions. It pays no specific attention to
representation. That is something you must do yourself in the
corresponding methods of the SQLData implementation.

>>> At least PostGIS currently uses a slightly different internal
>>> representation internally compared to what send/receive use. The
>>> internal on-disk format is optimized, the external representation is an
>>> upwards compatible extension to the OpenGIS standardized WKB format.
>>>
>>>
>> OK, so if you want to read that in Java, I assume your SQLData
>> input/outpout methods must deal with that. Perhaps I miss the point
>> altogether here. Who converts between the internal format and the
>> on-disk format?
>>
>
> For PostGIS: the send and receive functions which are given at CREATE
> DATATYPE.
>
>
Well, yes. But if you want to use the data types in you'll need to have
corresponding functions there. Are you suggesting that when reading, the
receive in the CREATE DATATYPE should be called first, and then, another
function should be called that would create the actual Java object from
the result of what the receive would output? IMHO, that's just pushing
the problem one step ahead. The SQLData implementation might just as
well mimic what the receive does in the first place, i.e. why not manage
the input to the receive rather then the output and save one roundtrip?

> http://svn.refractions.net/postgis/trunk/lwgeom/lwgeom_inout.c has the C
> code for those functions, LWGEOM_recv() and LWGEOM_send() which call the
> conversion routines WKBFromLWGEOM and LWGEOMFromWKB under the hood.
>
>
OK, so it would be fully possible to create special purpose Java/C JNI
mappings for that. But what we are discussing here is a *generic* way to
map virtually everything. You can do that by mimicing what lwgeom_inout
is doing in Java. If you don't want to mimic that, and if you don't want
special purpose JNI functions, then I have a hard time understanding how
you'd go about doing the mapping.

>
>>> So, from my understanding of the code, it's currently impossible to
>>> implement an 1:1 replacement for PostGIS in pljava. (Not that I
>>> seriously plan to do this.)
>>>
>> Not sure I understand why. Why can the data conversion not take place in
>> Java, should you choose to do that?
>>
>
> Because there is nobody that calls the appropriate Java functions.
>
>
On the contrary. If you map a PostGIS datatype to a SQLData
implementation class, PL/Java will see to that an instance of that class
is created and that it is fed with raw data from the PostGIS data type.

> When an SQL COPY reads binary data, it is passed through the RECEIVE
> function in UDT.c, and then put on the platters.
>
>
Right, and in that case you never see Java do anything. Not unless the
type is a UDT that is fully Java of course (with a CREATE DATATYPE that
actually appoints the functions in UDT.c)

> I hope that I managed to explain the problem, if not, please ask.
>
>
Still unclear I'm afraid :-)

Regards,
Thomas Hallgren