Re: invalidly encoded strings

From: Tatsuo Ishii <ishii(at)postgresql(dot)org>
To: pgsql(at)j-davis(dot)com
Cc: ishii(at)postgresql(dot)org, tgl(at)sss(dot)pgh(dot)pa(dot)us, andrew(at)dunslane(dot)net, laurenz(dot)albe(at)wien(dot)gv(dot)at, pgsql-hackers(at)postgresql(dot)org
Subject: Re: invalidly encoded strings
Date: 2007-09-11 02:53:06
Message-ID: 20070911.115306.35688137.t-ishii@sraoss.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches

> On Tue, 2007-09-11 at 11:27 +0900, Tatsuo Ishii wrote:
> > > BTW, it strikes me that there is another hole that we need to plug in
> > > this area, and that's the convert() function. Being able to create
> > > a value of type text that is not in the database encoding is simply
> > > broken. Perhaps we could make it work on bytea instead (providing
> > > a cast from text to bytea but not vice versa), or maybe we should just
> > > forbid the whole thing if the database encoding isn't SQL_ASCII.
> >
> > Please don't do that. It will break an usefull use case of convert().
> >
> > A user has a database encoded in UTF-8. He has English, French,
> > Chinese and Japanese data in tables. To sort the tables in the
> > language order, he will do like this:
> >
> > SELECT * FROM japanese_table ORDER BY convert(japanese_text using utf8_to_euc_jp);
> >
> > Without using convert(), he will get random order of data. This is
> > because Kanji characters are in random order in UTF-8, while Kanji
> > characters are reasonably ordered in EUC_JP.
>
> Isn't the collation a locale issue, not an encoding issue? Is there a
> ja_JP.UTF-8 that defines the proper order?

I don't think it helps. The point is, he needs different language's
collation, while PostgreSQL allows only one collation(locale) per
database cluster.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Dunstan 2007-09-11 02:54:20 Re: invalidly encoded strings
Previous Message Jeff Davis 2007-09-11 02:42:41 Re: invalidly encoded strings

Browse pgsql-patches by date

  From Date Subject
Next Message Andrew Dunstan 2007-09-11 02:54:20 Re: invalidly encoded strings
Previous Message Jeff Davis 2007-09-11 02:42:41 Re: invalidly encoded strings