Re: confusing / inefficient "need_transcoding" handling in copy

On Thu, Feb 08, 2024 at 05:25:01PM +0900, Sutou Kouhei wrote:
> In <20240206222445(dot)hzq22pb2nye7rm67(at)awork3(dot)anarazel(dot)de>
> "Re: confusing / inefficient "need_transcoding" handling in copy" on Tue, 6 Feb 2024 14:24:45 -0800,
> Andres Freund <andres(at)anarazel(dot)de> wrote:
>
>> One unfortunate issue: We don't have any tests verifying that COPY FROM
>> catches encoding issues.
>
> How about the attached patch for it?
>
> +CREATE TABLE test (t text);
> +COPY test FROM stdin WITH (ENCODING 'EUC_JP');
> +こんにちは
> +\.
> +
> +DROP TABLE test;

We have a couple of non-ASCII characters in the tests, but I suspect
that this one will not be digested correctly everywhere, even if
EUC_JP should be OK to use for the check. How about writing an
arbitrary sequence of bytes into a temporary file that gets used for
the COPY FROM instead? See for example how we do that with
abs_builddir in copy.sql.
--
Michael