Re: loading file with en dash character into postgres 9.6.1 database

Thanks Laurenz, that nailed it. It was what Tom was saying, except I didn't figure out how.

Thanks,
Patricia

-----Original Message-----
From: Albe Laurenz [mailto:laurenz(dot)albe(at)wien(dot)gv(dot)at]
Sent: Wednesday, July 12, 2017 5:31 AM
To: 'Tom Lane *EXTERN*'; Hu, Patricia
Cc: pgsql general (pgsql-general(at)postgresql(dot)org)
Subject: [EXTERNAL] RE: [GENERAL] loading file with en dash character into postgres 9.6.1 database

Tom Lane wrote:
> "Hu, Patricia" <Patricia(dot)Hu(at)finra(dot)org> writes:
>> The server and client encoding are both set to UTF8, and according to
>> this http://www.fileformat.info/info/unicode/char/2013/index.htm en
>> dash is a valid UTF8 character, but when running a script with insert
>> statement with en dash character in it, I got the error below.
>> psql:activity_type.lst:379: ERROR: invalid byte sequence for
>> encoding "UTF8": 0x96
>
> Well, that certainly isn't valid UTF8, so your script file isn't in UTF8.
>
>> If I set client_encoding to WIN1252, the same file will be run ok but
>> afterwards the en dash character showed up as "û", instead of the en
>> dash character "-"
>
> This indicates that your terminal program does *not* think its
> encoding is WIN1252. Having loaded that script file, you need to
> revert client_encoding to whatever your terminal program is using, or
> non-ASCII characters are going to be displayed wrong.
>
> A bit of poking around suggests that your terminal may be operating
> with code page 437 or similar, as 0x96 is "û" in that encoding ---
> according to Wikipedia, at least:
> https://en.wikipedia.org/wiki/Code_page_437
> I don't think Postgres supports that as a client_encoding setting, so
> one way or another you're going to need to switch the terminal
> program's character set setting.

Running "chcp 1252" in your Windows console before starting psql should do the trick.

Yours,
Laurenz Albe

Confidentiality Notice:: This email, including attachments, may include non-public, proprietary, confidential or legally privileged information. If you are not an intended recipient or an authorized agent of an intended recipient, you are hereby notified that any dissemination, distribution or copying of the information contained in or transmitted with this e-mail is unauthorized and strictly prohibited. If you have received this email in error, please notify the sender by replying to this message and permanently delete this e-mail, its attachments, and any copies of it immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.