From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> |
Cc: | anyhowstep(at)hotmail(dot)com, PostgreSQL mailing lists <pgsql-bugs(at)lists(dot)postgresql(dot)org> |
Subject: | Re: BUG #16281: LN() function inaccurate at 1000th fractional digit |
Date: | 2020-02-29 19:12:39 |
Message-ID: | 13048.1583003559@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-bugs |
Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> writes:
> Looking more closely at ln_var(), it seems that there was an oversight
> related to the way that it computes the local_rscale for the Taylor
> series expansion --- it fails to account for the fact that the result
> is multiplied by fact (2^(nsqrt+1), where nsqrt is the number of
> square roots performed in the range reduction phase, which in practice
> is at most 22).
> Since 2^22 has 7 decimal digits, multiplying by 2^22 almost entirely
> wipes out the 8-digit safety margin used in the Taylor series
> expansion. The attached patch corrects that.
Looks sane to me.
> With this patch, all the examples originally posted return the correct
> results (calculated with bc). I'd be interested to know how the OP
> constructed these examples, and whether there were any that were off
> by more than 1 ULP.
Yeah, that would be interesting.
regards, tom lane
From | Date | Subject | |
---|---|---|---|
Next Message | Justin AnyhowStep | 2020-03-01 02:59:38 | Re: BUG #16281: LN() function inaccurate at 1000th fractional digit |
Previous Message | Dean Rasheed | 2020-02-29 17:16:02 | Re: BUG #16281: LN() function inaccurate at 1000th fractional digit |