From: | "Joel Jacobson" <joel(at)compiler(dot)org> |
---|---|
To: | "Dean Rasheed" <dean(dot)a(dot)rasheed(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: Incorrect results from numeric round() and trunc() |
Date: | 2024-07-07 23:40:31 |
Message-ID: | c1ab12df-1357-4469-9761-9355015c039b@app.fastmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Sun, Jul 7, 2024, at 13:28, Dean Rasheed wrote:
> I've also tidied up a bit by replacing all instances of SHRT_MAX with
> a new constant NUMERIC_WEIGHT_MAX, whose name more accurately
> describes the limit, as used in various other overflow checks.
Having thought a bit more on this, I think we probably need a
DEC_DIGITS sensitive definition of NUMERIC_WEIGHT_MAX,
since per spec the max range for numeric is 0x20000 (131072)
decimal digits.
Therefore, I think perhaps what we want is:
+#define NUMERIC_DSCALE_MIN 0
+#define NUMERIC_WEIGHT_MAX ((0x20000/DEC_DIGITS)-1)
+#define NUMERIC_WEIGHT_MIN (-(NUMERIC_DSCALE_MAX+1)/DEC_DIGITS)
Maybe also 0x20000 (131072) should be a defined constant.
Regards,
Joel
From | Date | Subject | |
---|---|---|---|
Next Message | Richard Guo | 2024-07-07 23:49:43 | Re: report a typo in comments of ComputeXidHorizonsResult |
Previous Message | Michael Paquier | 2024-07-07 23:14:20 | Re: Pluggable cumulative statistics |