Re: Reporting UnicodeEncodeError info on arbitrary data sent to PG with psycopg3

Hello,

On Wed, 14 Feb 2024 at 16:42, Karl O. Pinc <kop(at)karlpinc(dot)com> wrote:

> I did not use conn.info.encoding because the docs say that it
> contains the _client_ encoding, not the server-side encoding
> used to store the db content.
> [...]
> Confirming the encodings, connecting to the "latin1" db with psql shows:

> kop_latin1=> show client_encoding;
> UTF8
>
> kop_latin1=> show server_encoding;
> LATIN1
>
> But, conn.info.encoding does return iso8859-1.
>
> So I think your documentation has confused client
> and server in this case. If you can confirm this
> for me I'll go ahead and use conn.info.encoding.

No, I am pretty sure that this is the client encoding that is
reported. It comes from here:

https://github.com/psycopg/psycopg/blob/ef6941df5b94997f79b429347c5d9b84e600bdd3/psycopg/psycopg/_encodings.py#L100-L101

which is a wrapper for PQparameterStatus:
https://www.postgresql.org/docs/current/libpq-status.html#LIBPQ-PQPARAMETERSTATUS
(so that the setting can be retrieved without running a query).

Maybe the way you are connecting via psql sets the client_encoding?
Can you try to get the result of `SHOW client_encoding` from psycopg?

From psycopg PoV, the client encoding is more important, because it's
how strings must be encoded to send them to the server; the server
encoding is relatively less important. So what you can actually store
is the smallest set of characters between server encoding and client
encoding. What you could do is to set the client encoding equal to the
server's:

SELECT set_config('client_encoding',
current_setting('server_encoding'), false);

and then proceed using `conn.info.encoding`.

Cheers

-- Daniele