Re: invalid byte sequence for encoding "UTF8": 0xff

----- Original Message -----
From: "Albe Laurenz" <all(at)adv(dot)magwien(dot)gv(dot)at>
To: "Ashish Karalkar *EXTERN*" <ashish(dot)karalkar(at)info-spectrum(dot)com>
Cc: <pgsql-general(at)postgresql(dot)org>
Sent: Monday, September 03, 2007 4:54 PM
Subject: RE: [GENERAL] invalid byte sequence for encoding "UTF8": 0xff

Ashish Karalkar wrote:
>>> I have a data script which runs fine from PgAdmin SQL
>>> Editor,but when I run this from command prompt I get
>>> following error:
>>>
>>> test=# \i /usr/local/pgsql/qsweb1/QSWEB_100_4_Default_Data.sql
>>>
>>> psql:/usr/local/pgsql/qsweb1/QSWEB_100_4_Default_Data.sql:1:
>>> ERROR: invalid byte sequence for encoding "UTF8": 0xff
>>>
>>> version
>>> --------------------------------------------------------------
>>> PostgreSQL 8.2.0 on i686-pc-linux-gnu, compiled by GCC gcc
>>> (GCC) 3.4.3 20041212 (Red Hat 3.4.3-9.EL4)
>>
>> - What is the client operating system (where you run psql and
>> PgAdmin III)?
>>
> Its Windows XP - PgAdmin III
> RHEL 3.4.3-9.EL4-psql (Server Machine)

So I get it that you run "psql" on Windows XP, right?

> PGCLIENTENCODING is not set and as per documantation I
> think by default it takes value of database i.e. UTF8
>
>> - What does the SQL command "show client_encoding;" return
>> when you issue it in
>> a) PgAdmin III
> UNICODE
>> b) psql
> UTF8

Ok, I suspect that's your problem.
You created QSWEB_100_4_Default_Data.sql by using the "Save" dialog
in PgAdmin III on the Windows machine, right?

Then the file will probably be encoded in Windows-1252.

If your client_encoding is set to UTF8, psql will expect UTF-8
data in the SQL script and complain if it meets wrong ones.

Does the script work as expected when you change the client
encoding to WIN1252?

>> - Please create a file that contains only the first line
>> of QSWEB_100_4_Default_Data.sql (I call it "l" in the following
>> commands), run the following two (Linux) commands on it:
>> a) od -t c l
>> b) od -t x1 l
>> and show us the output of both commands.
>
> [root(at)localhost qsweb]# od -t c test.sql
> 0000000 \ s e t O N _ E R R O R _ S
T
> 0000020 O P
> 0000022
> [root(at)localhost qsweb]# od -t x1 test.sql
> 0000000 5c 73 65 74 20 4f 4e 5f 45 52 52 4f 52 5f 53 54
> 0000020 4f 50
> 0000022

That's weird, because psql complained about line 1.

Maybe you messed something up by extracting the first line.

Try the following:

- Use binary file transfer and transfer the SQL script to a Linux
machine.

- Run "od -t c -t x1" on the file

- Find the 0xff that psql complains about.

Maybe that helps to locate the problem.
0xff is an unusual Windows-1252 character as well...

Hey Thanks Albe it worked.

Yours,
Laurenz Albe