Re: \COPY to accept non UTF-8 chars in CHAR columns

From: Andrew Gierth <andrew(at)tao11(dot)riddles(dot)org(dot)uk>
To: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Matthias Apitz <guru(at)unixarea(dot)de>, "pgsql-generallists(dot)postgresql(dot)org" <pgsql-general(at)lists(dot)postgresql(dot)org>
Subject: Re: \COPY to accept non UTF-8 chars in CHAR columns
Date: 2020-03-27 21:58:32
Message-ID: 87k135a2hd.fsf@news-spur.riddles.org.uk
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

>>>>> "Thomas" == Thomas Munro <thomas(dot)munro(at)gmail(dot)com> writes:

Thomas> Something like this approach might be useful for fixing the CSV file:

Thomas> https://codereview.stackexchange.com/questions/185821/convert-a-mix-of-latin-1-and-utf-8-to-proper-utf-8

Or:

perl -MEncode -pe '
use bytes;
sub c { decode("UTF-8",shift,sub { decode("windows-1252", chr(shift)) }); }
s/([\x80-\xFF]+)/encode("UTF-8",c($1))/eg' <infile >outfile

--
Andrew (irc:RhodiumToad)

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Justin King 2020-03-27 22:10:03 Re: PG12 autovac issues
Previous Message Thomas Munro 2020-03-27 20:40:30 Re: \COPY to accept non UTF-8 chars in CHAR columns