Re: Concerning about Unicode-aware string handling

On Mon, May 21, 2012 at 02:44:45AM -0700, John R Pierce wrote:
> support the bastardized UTF-16 'unicode' implemented by Windows NT

To be fair to Microsoft, while the BOM might be an irritant, they do
use a perfectly legitimate encoding of Unicode. There is no Unicode
requirement that code points be stored as UTF-8, and there is a strong
argument to be made that, for some languages, UTF-8 is extremely
inefficient and therefore the least preferred encoding. (Microsoft's
dependence on the BOM with UTF-16 -- really UCS2 -- is problematic, of
course, and appears to be adjusted in funny ways in Win 7.)

Because all wire protocols from the IETF use UTF-8 for Unicode
encoding, your best bet is still UTF-8 for maximal portability, so
your point about needing to make the database encoding and client
locale UTF-8 is correct.

Best,

A

--
Andrew Sullivan
ajs(at)crankycanuck(dot)ca