Re: OCTET_LENGTH is wrong

Tom Lane wrote:

>Barry Lind <barry(at)xythos(dot)com> writes:
>
>>While the text datatypes have additional issues with encodings, that is
>>not true for the bytea type. I think it does make sense that a client
>>be able to get the size in bytes that the bytea type value will return
>>to the client.
>>
>
>bytea does that already. It's only text that has (or had, till a few
>minutes ago) the funny behavior.
>
>I'm not set on the notion that octet_length should return on-disk size;
>that's clearly not what's contemplated by SQL92, so I'm happy to agree
>that if we want that we should add a new function to get it.
>("storage_length", maybe.) What's bothering me right now is the
>difference between client and server encodings. It seems that the only
>plausible use for octet_length is to do memory allocation on the client
>side,
>
Allocating memory seems for me to be drivers (libpq, JDBC, ODBC,...)
problem and
not something to be done by client code beforehand - at least for libpq
(AFAIK) we
don't have any means of giving it a pre-allocated storage area for one
field.

There is enough information in wire protocol for allocating right-sized
chunks at the
time query result is read. An additional call of "SELECT
OCTET_LENGTH(someCol)"
seems orders of magnitude slower than doing it at the right time in the
driver .

>and for that purpose the length ought to be measured in the client
>encoding. People seem to be happy with letting octet_length take the
>easy way out (measure in the server encoding), and I'm trying to get
>someone to explain to me why that's the right behavior. I don't see it.
>
perhaps we need another function "OCTET_LENGTH(someCol, encoding)" for
getting what we want and also client_encoding() and server_encoding()
for supplying
it some universal defaults ?

OTOH, from reading on Unicode I've came to a conlusion that there are
often several
ways for expressing the same string in Unicode, so for server encoding
not unicode and
client requesting unicode (say UTF-8) there can be several different
ways to express
the same string. Thus there is no absolute OCTET_LENGTH for
client_encoding for
all cases. Thus giving the actual uncompressed length seems most reasonable.

For unicode both in backend and frontend we could also make OCTET_LENGTH
return not int but an integer-interval of shortest and longest possible
encoding ;)

------------------
Hannu