Re: [PATCH] Fix old thinko in formula to compute sweight in numeric_sqrt().

From: "Joel Jacobson" <joel(at)compiler(dot)org>
To: "Dean Rasheed" <dean(dot)a(dot)rasheed(at)gmail(dot)com>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] Fix old thinko in formula to compute sweight in numeric_sqrt().
Date: 2023-01-31 21:59:10
Message-ID: 38ea7613-759d-4a54-91d3-ea1ffef31b17@app.fastmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Jan 31, 2023, at 20:25, Dean Rasheed wrote:
> That seems a bit wordy, given the context of this comment. I think
> it's sufficient to just give the formula, and note that it simplifies
> when DEC_DIGITS is even (not just 4):
>
> /*
> * Assume the input was normalized, so arg.weight is accurate. The result
> * then has at least sweight = floor(arg.weight * DEC_DIGITS / 2 + 1)
> * digits before the decimal point. When DEC_DIGITS is even, we can save
> * a few cycles, since the division is exact and there is no need to
> * round down.
> */
> #if DEC_DIGITS == ((DEC_DIGITS / 2) * 2)
> sweight = arg.weight * DEC_DIGITS / 2 + 1;
> #else
> if (arg.weight >= 0)
> sweight = arg.weight * DEC_DIGITS / 2 + 1;
> else
> sweight = 1 - (1 - arg.weight * DEC_DIGITS) / 2;
> #endif

Nice, you managed to simplify it even further.
I think the comment and the code now are crystal clear together.

I've tested it successfully, test report attached. In summary:
DEC_DIGITS=1 now produce 16 sig. figs. in the range sqrt(2e-31) .. sqrt(2e+32), which before had a mix of 17 and 18 sig. figs in the result.
DEC_DIGTIS=2 now produce 16 sig. figs. in the range sqrt(2e-31) .. sqrt(2e+31), which before always had 17 sig. figs in the result.
DEC_DIGITS=4 is unchanged.

Exact tested patch attached, code copy/pasted verbatim from your email.

Test

Attachment Content-Type Size
fix-sweight-v4.patch application/octet-stream 1020 bytes
fix-sweight-v4-test-output.txt text/plain 34.8 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2023-01-31 22:38:04 Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE
Previous Message David Rowley 2023-01-31 21:56:31 Re: heapgettup() with NoMovementScanDirection unused in core?