Re: Optimize mul_var() for var1ndigits >= 8

On Tue, Aug 13, 2024, at 00:56, Dean Rasheed wrote:
> On Mon, 12 Aug 2024 at 16:17, Joel Jacobson <joel(at)compiler(dot)org> wrote:
>>
>> On Mon, Aug 12, 2024, at 17:14, Joel Jacobson wrote:
>> > The case found with the smallest rscale adjustment was this one:
>> > -[ RECORD 1 ]------+--------------------------------
>> > var1 | 0.0000000000009873307197037692
>> > var2 | 0.426697279270850
>> > rscale_adjustment | -15
>> > expected | 0.0000000000004212913318381285
>> > numeric_mul_rscale | 0.0000000000004212913318381284
>> > diff | -0.0000000000000000000000000001
>>
>
> Hmm, interesting example. There will of course always be cases where
> the result isn't exact, but HEAD does produce the expected result in
> this case, and the intention is to always produce a result at least as
> accurate as HEAD, so it isn't working as expected.
>
..
> Updated patch attached, which fixes the above example and all the
> other differences produced by your test. I think, with a little
> thought, it ought to be possible to produce examples that round
> incorrectly in a more systematic (less brute-force) way. It should
> then be possible to construct examples where the patch differs from
> HEAD, but hopefully only by being more accurate, not less.

I reran the tests and v5 produces much fewer diffs than v4.
Not sure if the remaining ones are problematic or not.

joel(at)Joels-MBP postgresql % ./test-mul_var-init.sh
HEAD is now at a67a49648d Rename C23 keyword
SET
DROP TABLE
CREATE TABLE
COPY 1413
DROP TABLE
CREATE TABLE
setseed
---------

(1 row)

INSERT 0 14130000
COPY 14130000

joel(at)Joels-MBP postgresql % ./test-mul_var-verify-v4.sh
HEAD is now at a67a49648d Rename C23 keyword
SET
DROP TABLE
CREATE TABLE
COPY 14130000
Expanded display is on.
-[ RECORD 1 ]------+--------------------------------
var1 | 0.0000000000009873307197037692
var2 | 0.426697279270850
rscale_adjustment | -15
expected | 0.0000000000004212913318381285
numeric_mul_rscale | 0.0000000000004212913318381284
diff | -0.0000000000000000000000000001

Expanded display is off.
diff | count
--------------+----------
0.000e+00 | 14114384
1.000e-108 | 1
1.000e-211 | 1
1.000e-220 | 2
1.000e-228 | 6
1.000e-232 | 2
1.000e-235 | 1
1.000e-28 | 13
1.000e-36 | 1
1.000e-51 | 2
1.000e-67 | 1
1.000e-68 | 1
1.000e-80 | 1
-1.000e-1024 | 2485
-1.000e-108 | 3
-1.000e-144 | 2520
-1.000e-16 | 2514
-1.000e-228 | 4
-1.000e-232 | 1
-1.000e-27 | 36
-1.000e-28 | 538
-1.000e-32 | 2513
-1.000e-48 | 2473
-1.000e-68 | 1
-1.000e-80 | 2494
-2.000e-16 | 2
(26 rows)

rscale_adjustment | count
-------------------+-------
-237 | 2
-235 | 1
-232 | 3
-229 | 2
-228 | 8
-218 | 1
-108 | 4
-77 | 1
-67 | 1
-51 | 2
-38 | 3
-36 | 1
-28 | 5
-22 | 42
-17 | 7
-16 | 14959
-15 | 574
(17 rows)

joel(at)Joels-MBP postgresql % ./test-mul_var-verify-v5.sh
HEAD is now at a67a49648d Rename C23 keyword
SET
DROP TABLE
CREATE TABLE
COPY 14130000
Expanded display is on.
-[ RECORD 1 ]------+-------------------------------
var1 | 0.0000000000000000489673392928
var2 | 6.713030439846337
rscale_adjustment | -15
expected | 0.0000000000000003287192392308
numeric_mul_rscale | 0.0000000000000003287192392309
diff | 0.0000000000000000000000000001

Expanded display is off.
diff | count
--------------+----------
0.000e+00 | 14129971
1.000e-1024 | 1
1.000e-144 | 1
1.000e-16 | 1
1.000e-211 | 1
1.000e-220 | 2
1.000e-228 | 5
1.000e-232 | 1
1.000e-235 | 1
1.000e-28 | 8
1.000e-32 | 2
1.000e-36 | 1
1.000e-51 | 2
1.000e-67 | 1
1.000e-68 | 1
1.000e-80 | 1
(16 rows)

rscale_adjustment | count
-------------------+-------
-237 | 1
-235 | 1
-232 | 1
-229 | 2
-228 | 4
-218 | 1
-77 | 1
-67 | 1
-51 | 2
-38 | 1
-36 | 1
-28 | 2
-17 | 4
-16 | 5
-15 | 2
(15 rows)

Regards,
Joel