| From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
|---|---|
| To: | Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> |
| Cc: | anyhowstep(at)hotmail(dot)com, PostgreSQL mailing lists <pgsql-bugs(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: BUG #16281: LN() function inaccurate at 1000th fractional digit |
| Date: | 2020-02-29 19:12:39 |
| Message-ID: | 13048.1583003559@sss.pgh.pa.us |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-bugs |
Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> writes:
> Looking more closely at ln_var(), it seems that there was an oversight
> related to the way that it computes the local_rscale for the Taylor
> series expansion --- it fails to account for the fact that the result
> is multiplied by fact (2^(nsqrt+1), where nsqrt is the number of
> square roots performed in the range reduction phase, which in practice
> is at most 22).
> Since 2^22 has 7 decimal digits, multiplying by 2^22 almost entirely
> wipes out the 8-digit safety margin used in the Taylor series
> expansion. The attached patch corrects that.
Looks sane to me.
> With this patch, all the examples originally posted return the correct
> results (calculated with bc). I'd be interested to know how the OP
> constructed these examples, and whether there were any that were off
> by more than 1 ULP.
Yeah, that would be interesting.
regards, tom lane
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Justin AnyhowStep | 2020-03-01 02:59:38 | Re: BUG #16281: LN() function inaccurate at 1000th fractional digit |
| Previous Message | Dean Rasheed | 2020-02-29 17:16:02 | Re: BUG #16281: LN() function inaccurate at 1000th fractional digit |