From: | Philip Warner <pjw(at)rhyme(dot)com(dot)au> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | "Romanenko Mikhail" <mikhail(at)angg(dot)ru>, pgsql-general(at)postgresql(dot)org, pgsql-hackers(at)postgresql(dot)org |
Subject: | Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2 |
Date: | 2000-08-07 16:15:45 |
Message-ID: | 3.0.5.32.20000808021545.028196a0@mail.rhyme.com.au |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general pgsql-hackers |
At 12:11 7/08/00 -0400, Tom Lane wrote:
>Philip Warner <pjw(at)rhyme(dot)com(dot)au> writes:
>> pjw=# select float8(float4(10.1));
>> float8
>> ------------------
>> 10.1000003814697
>> (1 row)
>
>> I would have expected the latter to be at worst 10.10000000000000 +/-
>> .00000000000001.
>
>float4 is good to about 7 decimal digits (24 mantissa bits) on
>IEEE-standard machines. Thus the above result is actually closer
>than you have any right to expect.
>
>Don't they teach people about float arithmetic in CS 101 anymore?
>
No idea. It's a couple of decades since I did it.
I wasn't complaining about the float4 accuracy; I was complaining about the
way it was converted to float8. It seems more intuitive to zero-extend base
10 zeros...
----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: (+61) 0500 83 82 81 | _________ \
Fax: (+61) 0500 83 82 82 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/
From | Date | Subject | |
---|---|---|---|
Next Message | Philip Warner | 2000-08-07 16:28:37 | Re: [HACKERS] Re: Trouble with float4 afterupgrading from 6.5.3 to 7.0.2 |
Previous Message | Thomas Lockhart | 2000-08-07 16:12:59 | Re: [HACKERS] Re: Trouble with float4 afterupgrading from 6.5.3 to 7.0.2 |
From | Date | Subject | |
---|---|---|---|
Next Message | Zeugswetter Andreas SB | 2000-08-07 16:15:59 | AW: Anyone particularly wedded to func_tlist mechanism? |
Previous Message | Thomas Lockhart | 2000-08-07 16:12:59 | Re: [HACKERS] Re: Trouble with float4 afterupgrading from 6.5.3 to 7.0.2 |