Re: Unicode problem again

Michael Fuhr wrote:
> > > ProgrammingError Error Value: ERROR: character 0xe28099 of
> > > encoding "UTF8" has no equivalent in "LATIN1" select distinct
> > [...]
> >
> > This is UNICODE 0x2019, a "right single quotation mark".
> >
> > This is a "Windows character" - the only non-UNICODE codepages I
> > know that contain this character are the Microsoft codepages.
> [...]
> >
> > > I have changed client_encoding to Latin1 to get over errors
> > > caused by having the database in UTF8 and users trying to
> > > enter special characters like £ signs.
> > >
> > > Unfortunately, it seems there are already UTF8 encodings in
> > > the DB that have no equivalent in Latin1 from before the change.
>
> Your input data seems to have a mix of encodings: sometimes you're
> getting pound signs in a non-UTF-8 encoding, but if characters like
> <U+2019 RIGHT SINGLE QUOTATION MARK> got into the database when
> client_encoding was set to UTF8 then at least some data must have
> been in UTF-8.

Sorry, but that's not true.
That character is 0x9s in WINDOWS-1252.

So it could have been that client_encoding was (correctly) set to WIN1252
and the quotation mark was entered as a single byte character.

Yours,
Laurenz Albe