From: | "Joel Jacobson" <joel(at)compiler(dot)org> |
---|---|
To: | "Dean Rasheed" <dean(dot)a(dot)rasheed(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: Incorrect results from numeric round() and trunc() |
Date: | 2024-07-07 14:22:48 |
Message-ID: | c65c7129-50c6-4850-872b-b561701f99bf@app.fastmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Sun, Jul 7, 2024, at 13:28, Dean Rasheed wrote:
> The numeric round() and trunc() functions clamp the scale argument to
> the range between +/- NUMERIC_MAX_RESULT_SCALE, which is +/- 2000.
> That's a long way short of the actual allowed range of type numeric,
> so they produce incorrect results when rounding/truncating more than
> 2000 digits before or after the decimal point. For example,
> round(1e-5000, 5000) returns 0 instead of 1e-5000.
>
> Attached is a patch fixing that, using the actual documented range of
> type numeric.
>
> I've also tidied up a bit by replacing all instances of SHRT_MAX with
> a new constant NUMERIC_WEIGHT_MAX, whose name more accurately
> describes the limit, as used in various other overflow checks.
>
> In doing so, I also noticed a comment in power_var() which claimed
> that ln_dweight could be as low as -SHRT_MAX (-32767), which is wrong.
> It can only be as small as -NUMERIC_DSCALE_MAX (-16383), though that
> doesn't affect the point being made in that comment.
>
> I'd like to treat this as a bug-fix and back-patch it, since the
> current behaviour is clearly broken.
Fix seems straightforward to me.
I agree it should be back-patched.
Regards,
Joel
From | Date | Subject | |
---|---|---|---|
Next Message | Noah Misch | 2024-07-07 14:49:44 | Re: Confine vacuum skip logic to lazy_scan_skip |
Previous Message | Bryan Green | 2024-07-07 13:44:56 | Re: Simplifying width_bucket_numeric() |