Re: float conversion / presentation problem ?

From: David Fetter <david(at)fetter(dot)org>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: John R Pierce <pierce(at)hogranch(dot)com>, weigelt(at)metux(dot)de, pgsql-bugs(at)postgresql(dot)org
Subject: Re: float conversion / presentation problem ?
Date: 2005-04-07 16:10:10
Message-ID: 20050407161010.GD32435@fetter.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

On Thu, Apr 07, 2005 at 01:02:50AM -0400, Tom Lane wrote:
> John R Pierce <pierce(at)hogranch(dot)com> writes:
> > Tom Lane wrote:
> >> You do realize that float4/real is only good to six decimal
> >> places on most platforms?
>
> > and importantly, are stored as binary fractions. 0.10000 decimal
> > is a repeating fraction in binary, its like .1100110011.... x 2^-2
> > or something as a FLOAT.
>
> Right, and conversely a fraction that's cut off at a certain number
> of base-2 digits often corresponds to a repeating pattern of base-10
> digits. I once had a clue about the exact laws for this, but it was
> many years ago :-(

As with base-10, any denominator that can't be expressed as a power of
the prime factors in the base results in a repeating number. So
basically any denominator that can't be expressed as 2^n turns into a
repeating binary doohicky.

Cheers,
D
--
David Fetter david(at)fetter(dot)org http://fetter.org/
phone: +1 510 893 6100 mobile: +1 415 235 3778

Remember to vote!

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message Michael Williamson 2005-04-07 19:36:27 BUG #1591: BETWEEN NULL AND NULL causes crash
Previous Message Michaela 2005-04-07 15:04:35 BUG #1590: Comparison Operation with Strings