Re: invalid UTF-8 via pl/perl

From: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: Hannu Krosing <hannu(at)2ndquadrant(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: invalid UTF-8 via pl/perl
Date: 2010-01-03 20:09:39
Message-ID: D3E7DCCB-9ACE-4D02-B684-594BF0B61529@kineticode.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Jan 3, 2010, at 11:54 AM, Andrew Dunstan wrote:

> There are two issues with this patch. First, how far if at all should it be backpatched? All the way, or 8.3, where we tightened the encoding rules, or not at all?

8.3 seems reasonable.

> Second, It produces errors like this:
>
> andrew=# select 'a' || invalid_utf_seq() || 'b';
> ERROR: invalid byte sequence for encoding "UTF8": 0xd0
> HINT: This error can also happen if the byte sequence does not
> match the encoding expected by the server, which is controlled by
> "client_encoding".
> CONTEXT: PL/Perl function "invalid_utf_seq"
> andrew=#
>
>
> That hint seems rather misleading. I'm not sure what we can do about it though. If we set the noError param on pg_verifymbstr() we would miss the error message that actually identified the bad data, so that doesn't seem like a good plan.

I'm sure I'm just revealing my ignorance here, but how is the hint misleading?

Best,

David

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Dunstan 2010-01-03 20:14:05 Re: invalid UTF-8 via pl/perl
Previous Message David E. Wheeler 2010-01-03 20:07:48 Re: PATCH: Add hstore_to_json()