Re: Optimize mul_var() for var1ndigits >= 8

On Mon, Aug 12, 2024, at 17:14, Joel Jacobson wrote:
> The case found with the smallest rscale adjustment was this one:
> -[ RECORD 1 ]------+--------------------------------
> var1 | 0.0000000000009873307197037692
> var2 | 0.426697279270850
> rscale_adjustment | -15
> expected | 0.0000000000004212913318381285
> numeric_mul_rscale | 0.0000000000004212913318381284
> diff | -0.0000000000000000000000000001

To avoid confusion, correction: I mean "largest", since rscale_adjustment is less than or equal to zero.

Here is a group by rscale_adjustment to get a better picture:

SELECT
rscale_adjustment,
COUNT(*)
FROM
test_numeric_mul_rscale,
numeric_mul_rscale(var1, var2, rscale_adjustment)
WHERE numeric_mul_rscale IS DISTINCT FROM expected
GROUP BY rscale_adjustment
ORDER BY rscale_adjustment;

rscale_adjustment | count
-------------------+-------
-237 | 2
-235 | 1
-232 | 3
-229 | 2
-228 | 8
-218 | 1
-108 | 4
-77 | 1
-67 | 1
-51 | 2
-38 | 3
-36 | 1
-28 | 5
-22 | 42
-17 | 7
-16 | 14959
-15 | 574
(17 rows)

Regards,
Joel