Re: Numeric x^y for negative x

On Tue, 29 Jun 2021 at 12:08, Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> wrote:
>
> Numeric x^y is supported for x < 0 if y is an integer, but this
> currently fails if y is outside the range of an int32
>

I've been doing some more testing of this, and I spotted another
problem with numeric_power().

This is what happens when raising 0.9999999999 to increasingly large
powers, which should decrease to zero:

exp 0.9999999999^exp
10000000000 0.3678794411530483
100000000000 0.[4 zeros]4539992973978489
1000000000000 0.[43 zeros]3720075957420456
10000000000000 0.[434 zeros]5075958643751518
20000000000000 0.[868 zeros]2576535615307575
21000000000000 0.[912 zeros]9584908195943232
22000000000000 0.[955 zeros]3565658653381070
23000000000000 0.[998 zeros]13
23100000000000 0.[1000 zeros]
23200000000000 0.[1000 zeros]
23300000000000 1.[1000 zeros] *** WRONG ***
30000000000000 1.[1000 zeros] *** WRONG ***
40000000000000 1.[1000 zeros] *** WRONG ***
50000000000000 1.[1000 zeros] *** WRONG ***
60000000000000 1.[1000 zeros] *** WRONG ***
70000000000000 ERROR: value overflows numeric format

The cases where it returns 1 are a trivial logic bug in the
local_rscale calculation in power_var() -- when it computes
local_rscale from rscale and val, it needs to do so before clipping
rscale to NUMERIC_MAX_DISPLAY_SCALE, otherwise it ends up setting
local_rscale = 0, and loses all precision.

I also don't think it should be throwing an overflow error here. Some
code paths through numeric_power() catch cases that would underflow,
and return zero instead, but not all cases are caught. There's a
similar overflow error with numeric_exp() for large negative inputs
(-5999 returns 0, but -6000 overflows).

It's arguable though that numeric power() and exp() (and mul() for
that matter) should never return 0 for finite non-zero inputs, but
instead should throw underflow errors, which would make them
compatible with their floating-point counterparts. I don't think
that's useful though, and it's more likely to break people's code for
no real benefit. No other numeric code throws underflow errors.

So I think we should just attempt to avoid all such overflow errors,
that are actually underflows, and return zero instead.

Regards,
Dean