Re: confusing / inefficient "need_transcoding" handling in copy

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: Sutou Kouhei <kou(at)clear-code(dot)com>
Cc: andres(at)anarazel(dot)de, tgl(at)sss(dot)pgh(dot)pa(dot)us, pgsql-hackers(at)postgresql(dot)org, ishii(at)sraoss(dot)co(dot)jp
Subject: Re: confusing / inefficient "need_transcoding" handling in copy
Date: 2024-12-13 03:03:45
Message-ID: Z1ukEe2d7ml6-oaZ@paquier.xyz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Dec 12, 2024 at 03:25:41PM +0900, Sutou Kouhei wrote:
> In <Z1fKrTkT-eIVAK7F(at)paquier(dot)xyz>
> "Re: confusing / inefficient "need_transcoding" handling in copy" on Tue, 10 Dec 2024 13:59:25 +0900,
> Michael Paquier <michael(at)paquier(dot)xyz> wrote:
>> Another one would be valid conversions back and forth. For example,
>> I recall that LATIN1 accepts any bytes and can apply a conversion to
>> UTF-8, so we could use it and expand a bit more the proposed tests?
>> Or something like that?
>
> OK. I've added valid cases too by using LATIN1 as you
> suggested.

I may have missed something but v3 does not use that for a valid
conversion?

>> Likely, this should be made conditional, based on the fact that the
>> database needs to be able to support utf8? There are a couple of
>> examples like that in the tree, based on the following SQL trick:
>> SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
>> \if :skip_test
>> \quit
>> \endif
>
> Thanks. I didn't notice the portability problem. I've added
> the skip trick.

Indeed, thanks.

> Oh! I didn't know the "XXX_1.out" feature.

You have missed the inclusion of an alternate output, which should be
something like that to bypass the test rather than failing:
--- /dev/null
+++ b/src/test/regress/expected/copyencoding_1.out
@@ -0,0 +1,7 @@
+--
+-- Test cases for COPY encoding
+--
+-- skip test if not UTF8 server encoding
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit

I guess that you have the file, forgot a `git add`.
--
Michael

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message jian he 2024-12-13 03:15:02 Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row
Previous Message Thomas Munro 2024-12-13 02:56:00 Re: connection establishment versus parallel workers