Re: Perl DBI converts UTF-8 again to UTF-8 before sending it to the server

From: Matthias Apitz <guru(at)unixarea(dot)de>
To: Christoph Moench-Tegeder <cmt(at)burggraben(dot)net>
Cc: pgsql-general(at)lists(dot)postgresql(dot)org
Subject: Re: Perl DBI converts UTF-8 again to UTF-8 before sending it to the server
Date: 2019-10-04 15:07:49
Message-ID: 20191004150749.GB3724@c720-r342378
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

El día viernes, octubre 04, 2019 a las 04:29:32p. m. +0200, Christoph Moench-Tegeder escribió:

> ## Matthias Apitz (guru(at)unixarea(dot)de):
>
> > my $text = "\xc3\xa4";
> > print "text: ".$text."\n";
>
> Your output is lying to you:
> you need a binmode(STDOUT, ':encoding(utf8)'), which will make this print
> "ä", and a utf8::decode($text), after which you get "ä". And when you
> pass that $text through DBD::Pg, it will still be an "ä". And when you
> change $text to "ä", it still works. Most other combinations will
> either fail with "\xc3\xa4" or "ä".

Thanks, but:

...
my $text = "ä \xc3\xa4";
print "text before STDOUT set to UTF-8: ".$text."\n";
binmode(STDOUT, ':encoding(utf8)');
print "text after STDOUT set to UTF-8: ".$text."\n";
utf8::decode($text);
print "text after utf8::decode(\$text): ".$text."\n";
$sth->execute($text, 1) or die $sth->errstr, "\n";

gives:

./utf8.pl
text before STDOUT set to UTF-8: ä
text after STDOUT set to UTF-8: ä ä
text after utf8::decode($text): ä ä
printf "select tstchar25::bytea from dbctest ;\n" | psql -Usisis -dnewsisis
tstchar25
------------------------------------------------------------
\xc3a420c383c2a4202020202020202020202020202020202020202020
(1 Zeile)

> Welcome to modern perl's handling of utf8. Cue "Everything is Fine" meme.

Seems so :-(

matthias

--
Matthias Apitz, ✉ guru(at)unixarea(dot)de, http://www.unixarea.de/ +49-176-38902045
Public GnuPG key: http://www.unixarea.de/key.pub

3. Oktober! Wir gratulieren! Der Berliner Fernsehturm wird 50
aus: https://www.jungewelt.de/2019/10-02/index.php

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Christoph Moench-Tegeder 2019-10-04 15:22:03 Re: Perl DBI converts UTF-8 again to UTF-8 before sending it to the server
Previous Message Andres Freund 2019-10-04 15:06:10 Re: Postgres 12: backend crashes when creating non-deterministic collation