Re: Easy way to convert a database from WIN1252 to UTF8?

On Thu, Jul 01, 2010 at 10:01:02AM -0700, Mike Christensen wrote:
> Yup, the problem is line 170 doesn't actually match up to the
> DB.dbs.out file line 170 (which is a blank line). I believe it means
> line 170 from the stdin pipe it was processing for the copy command.

Doh, that's annoying. It would be nice to know that it's done the right
thing rather than "some" thing.

> Suffice to say, there was some weird character in my database that PG
> can't automatically translate from WIN1252 to UTF8, and apparently it
> will drop that /entire/ COPY command (the entire table doesn't get
> populated!)..

Yup, this is deliberate. You can also run psql with "-1" to put the
whole lot (i.e. every table/view/... creation and data insert) in a
transaction which will cause the whole restore to be rolled back if
something doesn't look right as well.

> As to what character was the culprit, I'm not entirely sure how to
> figure this out. I guess I could look for that hex value? However,
> if I set the encoding in the script itself, everything works
> perfectly.

PG is doing the right thing, 9D is undefined in Win1252. I guess you've
either got other problems or this was just an artifact of converting
from Win1252 to UTF8 external to PG and then not telling it that you'd
done that.

--
Sam http://samason.me.uk/