Re: [PATCH] Fix old thinko in formula to compute sweight in numeric_sqrt().

From: Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
To: Joel Jacobson <joel(at)compiler(dot)org>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] Fix old thinko in formula to compute sweight in numeric_sqrt().
Date: 2023-01-31 19:25:37
Message-ID: CAEZATCUhSn_1aQ63qguMF=4WmvqYirjM2nC8rPFjz2VAdHmcmQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, 31 Jan 2023 at 15:05, Joel Jacobson <joel(at)compiler(dot)org> wrote:
>
> I also think the performance impact no matter how small isn't worth it,
> but a comment based on your comments would be very valuable IMO.
>
> Below is an attempt at summarising your text, and to avoid the performance impact,
> maybe an #if so we get the general correct formula for DEC_DIGITS 1 or 2,
> and the reduced hand-optimised form for DEC_DIGITS 4?
> That could also improve readabilty, since readers perhaps more easily would see
> the relation between sweight and arg.weight, for the only DEC_DIGITS case we care about.
>

That seems a bit wordy, given the context of this comment. I think
it's sufficient to just give the formula, and note that it simplifies
when DEC_DIGITS is even (not just 4):

/*
* Assume the input was normalized, so arg.weight is accurate. The result
* then has at least sweight = floor(arg.weight * DEC_DIGITS / 2 + 1)
* digits before the decimal point. When DEC_DIGITS is even, we can save
* a few cycles, since the division is exact and there is no need to
* round down.
*/
#if DEC_DIGITS == ((DEC_DIGITS / 2) * 2)
sweight = arg.weight * DEC_DIGITS / 2 + 1;
#else
if (arg.weight >= 0)
sweight = arg.weight * DEC_DIGITS / 2 + 1;
else
sweight = 1 - (1 - arg.weight * DEC_DIGITS) / 2;
#endif

Regards,
Dean

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2023-01-31 20:22:02 Re: MacOS: xsltproc fails with "warning: failed to load external entity"
Previous Message Greg Stark 2023-01-31 19:16:06 Re: Timeline ID hexadecimal format