Bug #943: Server-Encoding from EUC_TW to UTF-8 doesn't work

From: pgsql-bugs(at)postgresql(dot)org
To: pgsql-bugs(at)postgresql(dot)org
Subject: Bug #943: Server-Encoding from EUC_TW to UTF-8 doesn't work
Date: 2003-04-10 12:11:35
Message-ID: 20030410121135.8A61947635D@postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs pgsql-hackers

Michael Enke (michael(dot)enke(at)wincor-nixdorf(dot)com) reports a bug with a severity of 2
The lower the number the more severe it is.

Short Description
Server-Encoding from EUC_TW to UTF-8 doesn't work

Long Description
System: SuSE Linux 8.1, kernel 2.4.19, glibc 2.2.5/glibc-locale 2.2.5
the same error on RedHat 7.3, kernel 2.4.20, glibc2.2.5
postgresql version 7.3.2
description: I loaded Chinese (TW) characters, encoded as UTF-8 into a
database which has UTF-8 encoding with "copy table from 'original'" with psql. Ok.
Than I exit from psql, exported PGCLIENTENCODING=EUC_TW
I started psql, make a "copy table to 'file.EUC_TW'". Ok.
If I convert this file to UTF-8 with iconv -f EUC-TW -t UTF-8 file.EUC_TW file.UTF-8
than file.UTF-8 looks ecaxtly the same as the original.
That means, PostgreSQL converts from UTF-8 to EUC_TW correct.
Now I load the exported file 'file.EUC_TW' back into DB:
"copy table2 from 'file.EUC_TW'", still I did not finish psql,
PGCLIENTENCODING is the same as for "copy to".
Now I get error telling me: "copy: line 1, LocalToUtf: could not convert (0xe5b5) EUC_TW to UTF-8" ... and the characters are missing in table2

Sample Code
UTF-8:
00000000: e795 b6e6 97a5 0ae5 959f e58b 95e4 b8ad
00000010: 2ce4 bd86 e69c 89e9 8caf e8aa a40a

EUC_TW as exported from PostgreSQL and not imported:
00000000: e5b5 c5ca 0ada f6d9 afc4 e32c c8fe c8b4
00000010: f2e3 eba8 0a

No file was uploaded with this report

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message pgsql-bugs 2003-04-10 19:13:35 Bug #944: implicit sequence creating error
Previous Message pgsql-bugs 2003-04-10 11:44:53 Bug #942: initdb hangs on install on w2k

Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas O'Dowd 2003-04-10 12:19:57 Re: Charset encoding and accents
Previous Message Peter Galbavy 2003-04-10 09:07:32 Re: More thoughts about FE/BE protocol