Re: UNICODE and SQL

On Monday 05 May 2003 22:24, Achilleus Mantzios wrote:
> On Mon, 5 May 2003, Ian Barwick wrote:
> > On Monday 05 May 2003 15:34, Marco Roda wrote:
> > > Hallo,
> > >
> > > I need to use SQL to insert some language specific characters into
> > > tables. In particular I am using German and Croatian specific
> > > characters. The database is created with UNICODE encoding.
> > > For instance, when trying to run from psql:
> > >
> > > INSERT INTO test VALUES (1,'Urlaubslite für nächstes Jahr');
> > >
> > > I get the following error:
> > >
> > > ERROR: Invalid UNICODE character sequence found (0xfc7220)
> > >
> > > because of 'ü' and 'ä'.
> >
> > What is your psql client encoding set to? Possibly you need
> > to set it to LATIN1.

or LATIN2 (?) for the Croatian characters.

> The UTF8 version of Latin1 is Latin1 itself,
> but german and iso8859-2 serbocroatian are
> non Latin1 (high ASCII) chars.

In PostgreSQL "LATIN1" is ISO 8859-1, see:
http://www.postgresql.org/docs/view.php?version=7.3&idoc=0&file=multibyte.html
Unicode Latin 1 (characters 160-255) happens to be the same as ISO 8859-1,
but in UTF-8 is represented as 2 bytes.

Setting psql's \encoding to LATIN1 will convert the client's 8-bit ISO 8859-1
characters (presuming this is the case) to UTF-8.

unitest=# \encoding unicode
unitest=# INSERT INTO test VALUES (1,'Urlaubslite für nächstes Jahr');
ERROR: Invalid UNICODE character sequence found (0xfc7220)
unitest=# \encoding latin1
unitest=# INSERT INTO test VALUES (1,'Urlaubslite für nächstes Jahr');
INSERT 19134 1
unitest=# select * from test;
id | val
----+-------------------------------
1 | Urlaubslite für nächstes Jahr
(1 row)
unitest=# \encoding unicode
unitest=# select * from test;
id | val
----+-------------------------------
1 | Urlaubslite für nächstes Jahr
(1 row)

Ian Barwick
barwick(at)gmx(dot)net