Re: Facing issue in using special characters

From: Gunther <raj(at)gusw(dot)net>
To: M Tarkeshwar Rao <m(dot)tarkeshwar(dot)rao(at)ericsson(dot)com>, pgsql-general <pgsql-general(at)postgresql(dot)org>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>, "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org>, "'pgsql-hackers-owner(at)postgresql(dot)org'" <pgsql-hackers-owner(at)postgresql(dot)org>
Subject: Re: Facing issue in using special characters
Date: 2019-03-15 15:59:48
Message-ID: c69d480d-89c4-f2c6-83bb-d0202785211b@gusw.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers pgsql-performance

This is not an issue for "hackers" nor "performance" in fact even for
"general" it isn't really an issue.

"Special characters" is actually nonsense.

When people complain about "special characters" they haven't thought
things through.

If you are unwilling to think things through and go step by step to make
sure you know what you are doing, then you will not get it and really
nobody can help you.

In my professional experience, people who complain about "special
characters" need to be cut loose or be given a chance (if they are
established employees who carry some weight). If a contractor complains
about "special characters" they need to be fired.

Understand charsets -- character set, code point, and encoding. Then
understand how encoding and string literals and "escape sequences" in
string literals might work.

Know that UNICODE today is the one standard, and there is no more need
to do code table switch. There is nothing special about a Hebrew alef or
a greek lower case alpha or a latin A. Nor a hyphen and en-dash or an
em-dash. All these characters are in the UNICODE. Yes, there are some
Japanese who claim that they don't like that their Chinese character
versions are put together with simplified reform Chinese font. But
that's a font issue, not a character code issue.

7 bit ASCII is the first page of UNICODE, even in the UTF-8 encoding.

ISO Latin 1, or the Windoze 123 whatever special table of ISO Latin 1
has the same code points as UNICODE pages 0 and 1, but not compatible
with UTF-8 coding because of the way UTF-8 uses the 8th bit.

But none of this is likely your problem.

Your problem is about string literals in SQL for examples. About the
configuration of your database (I always use initdb with --locale C and
--encoding UTF-8). Use UTF-8 in the database. Then all your issues are
about string literals in SQL and in JAVA and JSON and XML or whatever
you are using.

You have to do the right thing. If you produce any representation,
whether that is XML or JSON or SQL or URL query parameters, or a CSV
file, or anything at all, you need to escape your string values properly.

This question with no detail didn't deserve such a thorough answer, but
it's my soap box. I do not accept people complaining about "special
characters". My own people get that same sermon from me when they make
that mistake.

-Gunther

On 3/15/2019 1:19, M Tarkeshwar Rao wrote:
>
> Hi all,
>
> Facing issue in using special characters. We are trying to insert
> records to a remote Postgres Server and our application not able to
> perform this because of errors.
>
> It seems that issue is because of the special characters that has been
> used in one of the field of a row.
>
> Regards
>
> Tarkeshwar
>

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Alexandru Lazarev 2019-03-15 16:02:02 jsonb_set performance degradation / multiple jsonb_set on multiple documents
Previous Message Jeremy Finzel 2019-03-15 15:16:27 Re: Do all superuser processes count toward superuser_reserved_connections?

Browse pgsql-hackers by date

  From Date Subject
Next Message Pavel Stehule 2019-03-15 16:00:54 Re: string_to_array, array_to_string function without separator
Previous Message Chapman Flack 2019-03-15 15:59:01 Re: string_to_array, array_to_string function without separator

Browse pgsql-performance by date

  From Date Subject
Next Message Alexandru Lazarev 2019-03-15 16:02:02 jsonb_set performance degradation / multiple jsonb_set on multiple documents
Previous Message David G. Johnston 2019-03-15 06:33:52 Re: Facing issue in using special characters