\COPY to accept non UTF-8 chars in CHAR columns

From: Matthias Apitz <guru(at)unixarea(dot)de>
To: pgsql-general(at)lists(dot)postgresql(dot)org
Subject: \COPY to accept non UTF-8 chars in CHAR columns
Date: 2020-03-27 15:36:23
Message-ID: 20200327153623.GA32266@sh4-5.1blu.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hello,

We're moving databases from Sybase/Oracle to PostgreSQL 11.4 by
unloading them with our own tool into a CSV like format and loading the
rows with \COPY. One can imagine that in old databases, in use for
a longer time, you will find any sort of dust which should not be there, for
example ISO-8859-1 chars while the CHAR column should be in UTF-8. Needless
to say, this data should not be there, it is just a fact, in parts
caused by foreign data loaded into the Sybase/Oracle database, sometimes
our own Java- or Perl-written software, breaks a longer CHAR column into
two sequential rows (with an INT column to note the order), but does
the break in the middle of an UTF-8 multi-byte, clumsy. :-(

In short, it there a way to let \COPY accept such broken ISO bytes, just
complaining about, but not stopping the insert of the row?

Thanks

matthias

--
Matthias Apitz, ✉ guru(at)unixarea(dot)de, http://www.unixarea.de/ +49-176-38902045
Public GnuPG key: http://www.unixarea.de/key.pub
May, 9: Спаси́бо освободители! Thank you very much, Russian liberators!

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2020-03-27 15:46:34 Re: \COPY to accept non UTF-8 chars in CHAR columns
Previous Message Tom Lane 2020-03-27 14:00:49 Re: Issue with Postgres process startup after instance restart