From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | Daniel Vázquez <daniel2d2art(at)gmail(dot)com> |
Cc: | pgsql-hackers(at)postgresql(dot)org |
Subject: | Re: Same double precision operations, different results |
Date: | 2014-02-13 17:26:48 |
Message-ID: | 6262.1392312408@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
=?ISO-8859-1?Q?Daniel_V=E1zquez?= <daniel2d2art(at)gmail(dot)com> writes:
> Please can someone tell me why this behavior? Thx!!
You're confusing numeric and double precision; in particular this
calculation is *not* being done in float8, but numeric:
> # select (0.766238989559398 * 0.766238989559398 * 1 + 0.642555686986733 *
> 0.642555686986733) calc;
> (*) Why this calculation produces 1 and not 0.999999999999999633651488135693
Actually, it's not producing 1, but a smidgen more:
regression=# set extra_float_digits TO 3;
SET
regression=# select ( cast (
cos(radians(39.9826557))
* cos(radians(39.9826557))
* cos(radians(-0.04773120000004383) - radians(-0.04773120000004383))
+ sin(radians(39.9826557))
* sin(radians(39.9826557)) as double precision )
);
float8
---------------------
1.00000000000000022
(1 row)
You've got roundoff error either way, but this way happens to be in the
direction that makes acos() complain.
regards, tom lane
From | Date | Subject | |
---|---|---|---|
Next Message | Alvaro Herrera | 2014-02-13 17:40:39 | Re: truncating pg_multixact/members |
Previous Message | Daniel Vázquez | 2014-02-13 17:16:07 | Same double precision operations, different results |