Re: Automatically parsing in-line composite types

Hi!

On Wed, Oct 30, 2019 at 8:37 AM Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:
> Check out libpqtypes: https://github.com/pgagarinov/libpqtypes

Interesting. I have looked at the code a bit but I do not find how it
determines the type for inline compound types, like the ones they
appear in my original SQL query example. Could you maybe point me to
the piece of code there handling that? Because to my
understanding/exploration that information is simply not exposed to
the client in any way. :-(

> it does exactly what you want. It's a wrapper for libpq that provides
> client side parsing for the binary protocol with array and composite
> type parsing.

It looks to me that it does parsing of composite types only if they
are registered composite types. But not for example ones you get if
you project a subset of fields from a table in a subquery. That has no
registered composite type?

Also, how you are handling discovery of registered types, do you read
that on-demand from the database? They are not provided over the wire?

> Virtually any
> non-C client application really ought to be using json rather than the
> custom binary structures libpqtyps would provide.

I thought that initially, too, but then found out that JSON has some
heavy limitations because the implementation in PostgreSQL is standard
based. There is also no hook to do custom encoding of non-JSON values.
So binary blobs are converted in an ugly way (base64 would be better).
You also loose a lot of meta-information, because everything non-JSON
gets converted to strings automatically. Like knowing what is a date.
I think MongoDB with BSON made much more sense here. It looks like
perfect balance between simplicity of JSON structure and adding few
more useful data types.

But yes, JSON is great also because clients often have optimized JSON
readers. Which can beat any other binary serialization format. In
node.js, it is simply the fastest there is to transfer data:

https://mitar.tnode.com/post/in-nodejs-always-query-in-json-from-postgresql/

Mitar

--
http://mitar.tnode.com/
https://twitter.com/mitar_m