Re: numeric and float converts to int differently?

From: Christopher Browne <cbbrowne(at)libertyrms(dot)info>
To: pgsql-sql(at)postgresql(dot)org
Subject: Re: numeric and float converts to int differently?
Date: 2003-10-28 23:11:13
Message-ID: 60u15tklf2.fsf@dev6.int.libertyrms.info
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-sql

surrano(at)mailbox(dot)hu (SZUCS Gábor) writes:
> As Tom pointed out, it isn't a floating point failure -- it is how rounding
> float is implemented. I assume anything with less than 15 digits can be
> exactly represented as float.

No, "decimal" fractions cannot ever be exactly represented in floating
point because since they use powers of two, you wind up with repeated
fractions.

1/3 is approximately 0.3333333, but you cannot present that exactly in
decimal.

In the very same way, 1/10 is approximately equal to
0.001001001001001001001001001001 (as a binary 'fraction'); the '001'
part is a repeating group, and wherever you terminate it, you lose
exactness.
--
(reverse (concatenate 'string "ofni.smrytrebil" "@" "enworbbc"))
<http://dev6.int.libertyrms.com/>
Christopher Browne
(416) 646 3304 x124 (land)

In response to

Browse pgsql-sql by date

  From Date Subject
Next Message Merrall, Graeme 2003-10-29 01:36:03 SOLVED: Emulating 'connect by prior' using stored proc
Previous Message Bruno Wolff III 2003-10-28 20:31:19 Re: numeric and float converts to int differently?