Re: Reporting UnicodeEncodeError info on arbitrary data sent to PG with psycopg3

From: "Karl O(dot) Pinc" <kop(at)karlpinc(dot)com>
To: Daniele Varrazzo <daniele(dot)varrazzo(at)gmail(dot)com>
Cc: "psycopg(at)postgresql(dot)org" <psycopg(at)postgresql(dot)org>
Subject: Re: Reporting UnicodeEncodeError info on arbitrary data sent to PG with psycopg3
Date: 2024-02-14 15:42:03
Message-ID: 20240214094203.52d7e22d@slate.karlpinc.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: psycopg

Hi Daniele,

On Wed, 14 Feb 2024 15:30:33 +0100
Daniele Varrazzo <daniele(dot)varrazzo(at)gmail(dot)com> wrote:

> Note however that if you just want to know the Python codec you can
> find it in `conn.info.encoding`
> (https://www.psycopg.org/psycopg3/docs/api/objects.html#psycopg.ConnectionInfo.encoding)
>
> >>> conn.info.encoding
> 'iso8859-1'
> >>> "€".encode(conn.info.encoding)
> ...
> UnicodeEncodeError: 'latin-1' codec can't encode character
> '\u20ac' in position 0: ordinal not in range(256)

Thanks very much for the help. Working directly with the encoding
of the server side, translated to python, is indeed a more
direct approach.

I did not use conn.info.encoding because the docs say that it
contains the _client_ encoding, not the server-side encoding
used to store the db content.

From the link above:

```
encoding

The Python codec name of the connection’s client encoding.

The value returned is always normalized to the Python codec name:

conn.execute("SET client_encoding TO LATIN9")
conn.info.encoding
'iso8859-15'
```

Confirming the encodings, connecting to the "latin1" db with psql shows:

```
$ psql -U kop latin1
psql (15.5 (Debian 15.5-0+deb12u1))
Type "help" for help.

kop_latin1=> show client_encoding;
client_encoding
-----------------
UTF8
(1 row)

kop_latin1=> show server_encoding;
server_encoding
-----------------
LATIN1
(1 row)
```

But, conn.info.encoding does return iso8859-1.

So I think your documentation has confused client
and server in this case. If you can confirm this
for me I'll go ahead and use conn.info.encoding.

Thanks again.

Regards,

Karl <kop(at)karlpinc(dot)com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

In response to

Responses

Browse psycopg by date

  From Date Subject
Next Message Daniele Varrazzo 2024-02-14 18:28:57 Re: Reporting UnicodeEncodeError info on arbitrary data sent to PG with psycopg3
Previous Message Daniele Varrazzo 2024-02-14 14:30:33 Re: Reporting UnicodeEncodeError info on arbitrary data sent to PG with psycopg3