From: | "Joel Jacobson" <joel(at)compiler(dot)org> |
---|---|
To: | "Dean Rasheed" <dean(dot)a(dot)rasheed(at)gmail(dot)com> |
Cc: | pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: [PATCH] Fix old thinko in formula to compute sweight in numeric_sqrt(). |
Date: | 2023-01-31 21:59:10 |
Message-ID: | 38ea7613-759d-4a54-91d3-ea1ffef31b17@app.fastmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Tue, Jan 31, 2023, at 20:25, Dean Rasheed wrote:
> That seems a bit wordy, given the context of this comment. I think
> it's sufficient to just give the formula, and note that it simplifies
> when DEC_DIGITS is even (not just 4):
>
> /*
> * Assume the input was normalized, so arg.weight is accurate. The result
> * then has at least sweight = floor(arg.weight * DEC_DIGITS / 2 + 1)
> * digits before the decimal point. When DEC_DIGITS is even, we can save
> * a few cycles, since the division is exact and there is no need to
> * round down.
> */
> #if DEC_DIGITS == ((DEC_DIGITS / 2) * 2)
> sweight = arg.weight * DEC_DIGITS / 2 + 1;
> #else
> if (arg.weight >= 0)
> sweight = arg.weight * DEC_DIGITS / 2 + 1;
> else
> sweight = 1 - (1 - arg.weight * DEC_DIGITS) / 2;
> #endif
Nice, you managed to simplify it even further.
I think the comment and the code now are crystal clear together.
I've tested it successfully, test report attached. In summary:
DEC_DIGITS=1 now produce 16 sig. figs. in the range sqrt(2e-31) .. sqrt(2e+32), which before had a mix of 17 and 18 sig. figs in the result.
DEC_DIGTIS=2 now produce 16 sig. figs. in the range sqrt(2e-31) .. sqrt(2e+31), which before always had 17 sig. figs in the result.
DEC_DIGITS=4 is unchanged.
Exact tested patch attached, code copy/pasted verbatim from your email.
Test
Attachment | Content-Type | Size |
---|---|---|
fix-sweight-v4.patch | application/octet-stream | 1020 bytes |
fix-sweight-v4-test-output.txt | text/plain | 34.8 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Andres Freund | 2023-01-31 22:38:04 | Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE |
Previous Message | David Rowley | 2023-01-31 21:56:31 | Re: heapgettup() with NoMovementScanDirection unused in core? |