Re: Same double precision operations, different results

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Daniel Vázquez <daniel2d2art(at)gmail(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Same double precision operations, different results
Date: 2014-02-13 17:26:48
Message-ID: 6262.1392312408@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

=?ISO-8859-1?Q?Daniel_V=E1zquez?= <daniel2d2art(at)gmail(dot)com> writes:
> Please can someone tell me why this behavior? Thx!!

You're confusing numeric and double precision; in particular this
calculation is *not* being done in float8, but numeric:

> # select (0.766238989559398 * 0.766238989559398 * 1 + 0.642555686986733 *
> 0.642555686986733) calc;

> (*) Why this calculation produces 1 and not 0.999999999999999633651488135693

Actually, it's not producing 1, but a smidgen more:

regression=# set extra_float_digits TO 3;
SET
regression=# select ( cast (
cos(radians(39.9826557))
* cos(radians(39.9826557))
* cos(radians(-0.04773120000004383) - radians(-0.04773120000004383))
+ sin(radians(39.9826557))
* sin(radians(39.9826557)) as double precision )
);
float8
---------------------
1.00000000000000022
(1 row)

You've got roundoff error either way, but this way happens to be in the
direction that makes acos() complain.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2014-02-13 17:40:39 Re: truncating pg_multixact/members
Previous Message Daniel Vázquez 2014-02-13 17:16:07 Same double precision operations, different results