pgsql: Fix backslash-escaping multibyte chars in COPY FROM.

From: Heikki Linnakangas <heikki(dot)linnakangas(at)iki(dot)fi>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Fix backslash-escaping multibyte chars in COPY FROM.
Date: 2021-02-05 09:58:58
Message-ID: E1l7xtC-00068D-5Z@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Fix backslash-escaping multibyte chars in COPY FROM.

If a multi-byte character is escaped with a backslash in TEXT mode input,
and the encoding is one of the client-only encodings where the bytes after
the first one can have an ASCII byte "embedded" in the char, we didn't
skip the character correctly. After a backslash, we only skipped the first
byte of the next character, so if it was a multi-byte character, we would
try to process its second byte as if it was a separate character. If it
was one of the characters with special meaning, like '\n', '\r', or
another '\\', that would cause trouble.

One such exmple is the byte sequence '\x5ca45c2e666f6f' in Big5 encoding.
That's supposed to be [backslash][two-byte character][.][f][o][o], but
because the second byte of the two-byte character is 0x5c, we incorrectly
treat it as another backslash. And because the next character is a dot, we
parse it as end-of-copy marker, and throw an "end-of-copy marker corrupt"
error.

Backpatch to all supported versions.

Reviewed-by: John Naylor, Kyotaro Horiguchi
Discussion: https://www.postgresql.org/message-id/a897f84f-8dca-8798-3139-07da5bb38728%40iki.fi

Branch
------
REL_11_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/d9b4163c553eb8ae47034946c11efb04e9498f6f

Modified Files
--------------
src/backend/commands/copy.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Robert Haas 2021-02-05 18:54:06 pgsql: Move some code from src/bin/scripts to src/fe_utils to permit re
Previous Message Etsuro Fujita 2021-02-05 06:35:43 pgsql: postgres_fdw: Fix assertion in estimate_path_cost_size().