Re: Import/Export failure due to UTF-8 error in pgAdmin4 but not in pgAdmin3

Hi Rik

On Tue, Jan 8, 2019 at 6:53 PM richard coleman
<rcoleman(dot)ascentgl(at)gmail(dot)com> wrote:
>
> Dave,
>
> Thanks for continuing this discussion, but I think you misunderstand the situation. I am storing valid non-UTF8 data in a SQL_ASCII encoded postgreSQL database (please re-read what I had previously written). This is why psql has NO problem dealing with it. This is also why Windows ODBC and .Net applications have NO problem dealing with it. In fact the most common character that pgAdmin4 crashes on is the Windows smart quote. So to reiterate, I am using valid non-UTF8 characters in a SQL_ASCII database. This is a supported configuration for postgreSQL. The issue seems to be that pgAdmin4 is assuming UTF8 data and crashing/failing/throwing errors when it encounters invalid UTF8 characters.
>
> I hope I have made the situation a little bit clearer.

Well psql is failing to deal with it *in this case*, as that's what is
doing the \copy in the import/export tool.

In other cases (i.e. the ones where pgAdmin sees the data, such as
results in the query tool), the issue arises because Python and/or
Javascript (and by extension pgAdmin) may barf on data encoded in a
way they don't recognise. That's why the PostgreSQL docs say to only
use ASCII data in SQL_ASCII databases - the behaviour is undefined,
and as a result may either not render properly or may crash or error
on non-ASCII data.

Anyhoo, I expect to have a little time after dinner shortly so I'll
try out the workaround I thought of earlier to see if it helps (I
doubt it'll be a panacea, but it may help in some cases).

By any chance do you have a test case you can share with me that
refuses to export from pgAdmin (using the Import/Export tool)? If so,
I'd appreciate a copy of it to play with.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company