Re: BUG #17946: LC_MONETARY & DO LANGUAGE plperl - BUG

From: Joe Conway <mail(at)joeconway(dot)com>
To: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Cc: Tristan Partin <tristan(at)neon(dot)tech>, gdo(at)leader(dot)it, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PostgreSQL-development <pgsql-hackers(at)postgreSQL(dot)org>
Subject: Re: BUG #17946: LC_MONETARY & DO LANGUAGE plperl - BUG
Date: 2023-06-20 22:02:48
Message-ID: 839d893b-1681-f6b6-e1ba-685c34c582ca@joeconway.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs pgsql-hackers

On 6/19/23 19:30, Heikki Linnakangas wrote:
> On 18/06/2023 21:27, Joe Conway wrote:
>> I have proposed a targeted fix that I believe is safe to backpatch --
>> attached.
>>
>> IIUC, Tom was +1, but Heikki was looking for a more general solution.
>>
>> My issue with the more general solution is that it will likely be too
>> invasive to backpatch, and at the moment at least, there are no other
>> confirmed bugs related to all of this (even if the current code is more
>> fragile than we would prefer).
>
> Ok, I agree switching to uselocale() everywhere is too much to
> backpatch. We should consider it for master though.

Makes sense

> With the patch you're proposing, do we now have a coding rule that you
> must call "uselocale(LC_GLOBAL_LOCALE)" before every and any call to
> setlocale()? If so, you missed a few spots: pg_perm_setlocale,
> pg_bind_textdomain_codeset, and cache_locale_time.

Well I was not proposing such a rule (trying to stay narrowly focused on
the demonstrated issue) but I suppose it might make sense. Anywhere we
use setlocale() we are depending on subsequent locale operations to use
the global locale. And uselocale(LC_GLOBAL_LOCALE) itself looks like it
ought to be pretty cheap.

> The current locale affects a lot of other things than localeconv()
> calls. For example, LC_MESSAGES affects all strerror() calls. Do we need
> to call "uselocale(LC_GLOBAL_LOCALE)" before all possible strerror()
> calls too?

That seems heavy handed

> I think we should call "uselocale(LC_GLOBAL_LOCALE)" immediately after
> returning from the perl interpreter, instead of before setlocale()
> calls, if we want all Postgres code to run with the global locale. Not
> sure how much performance overhead that would have.

I don't see how that is practical, or at least it does not really
address the issue. I think any loaded shared library could cause the
same problem by running newlocale() + uselocale() on init. Perhaps I
should go test that theory though.

> I just found out about perl's "switch_to_global_locale" function
> (https://perldoc.perl.org/perlapi#switch_to_global_locale) Should we
> use that?

Maybe, although it does not seem to exist on the older perl version on
RHEL7. And same comment as above -- while it might solve the problem
with libperl, it doesn't address similar problems with other loaded
shared libraries.

> Testing the patch, I bumped into this:
>
> postgres=# create or replace function finnish_to_number() returns
> numeric as $$ select to_number('1,23', '9D99'); $$ language sql set
> lc_numeric to 'fi_FI.utf8';
> CREATE FUNCTION
> postgres=# DO LANGUAGE 'plperlu' $$
> use POSIX qw(setlocale LC_NUMERIC);
> use locale;
>
> setlocale LC_NUMERIC, "fi_FI.utf8";
>
> $n = 5/2; # Assign numeric 2.5 to $n
>
> spi_exec_query('SELECT finnish_to_number()');
>
> $a = " $n"; # Locale-dependent conversion to string
> elog(NOTICE, "half five is $n"); # Locale-dependent output
> $$;
> NOTICE: half five is 2,5
> DO
> postgres=# select to_char(now(), 'Day');
> WARNING: could not determine encoding for locale "en_GB.UTF-8": codeset
> is "ANSI_X3.4-1968"
> to_char
> -----------
> Tuesday
> (1 row)

Do you think that is because uselocale(LC_GLOBAL_LOCALE) pulls out the
rug from under perl?

--
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Thomas Munro 2023-06-20 22:54:15 Re: BUG #17949: Adding an index introduces serialisation anomalies.
Previous Message Tom Lane 2023-06-20 20:59:50 Re: BUG #17985: Inconsistent results of SELECT comparing two CASE WHEN clause

Browse pgsql-hackers by date

  From Date Subject
Next Message Nathan Bossart 2023-06-20 22:20:24 Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX
Previous Message Jeff Davis 2023-06-20 20:46:29 Re: collation-related loose ends before beta2