Re: libpq Unicode support?

Are we not going to lose some characters if we are putting a UTF-16 to UTF-8
translation in front of libpq?

Ale.

-----Original Message-----
From: Tom Lane [mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us]
Sent: Friday, April 22, 2005 12:14 PM
To: Bruce Momjian
Cc: Ale Raza; pgsql-general(at)postgresql(dot)org
Subject: Re: [GENERAL] libpq Unicode support?

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Tom Lane wrote:
>> Oh? Who's working on it, or even interested? Was there discussion
>> of adding it to TODO?

> TODO has:

> o Add support for Unicode

> To fix this, the data needs to be converted to/from UTF16/UTF8
> so the Win32 wcscoll() can be used, and perhaps other functions
> like towupper(). However, UTF8 already works with normal
> locales but provides no ordering or character set classes.

That's completely unrelated --- it's talking about making correct use of
Windows' locale support in one small bit inside the server.

To make libpq UTF-16 capable, we'd have to change its API for all
strings; either make the strings counted rather than null-terminated,
or make the string elements wchar instead of char. After that we'd
have to hack the FE/BE protocol too (or more likely, require libpq
to translate UTF-16 to UTF-8 before sending to the server). I don't
foresee anyone doing any of this, at least not in the near term.

Putting a UTF-16 to UTF-8 translation in front of libpq seems a lot
more practical.

regards, tom lane