Re: somebody could explain this?

From: Martijn van Oosterhout <kleptog(at)svana(dot)org>
To: Cristian Prieto <cristian(at)clickdiario(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: somebody could explain this?
Date: 2005-11-04 16:45:43
Message-ID: 20051104164542.GC13966@svana.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Nov 04, 2005 at 10:16:50AM -0600, Cristian Prieto wrote:
> Hello, I'm using PostgreSQL 8.0.4 in Fedora Core 3, right now I'm learning a
> little about the postgresql internals and the way some kind of SPs could be
> written in c language; I found something really weird and I cannot explain
> to me this behavior:

What's happening here is that the multiplication, being floating point,
has some accumulated error such that when you multiply it by 100 and
convert it to an int, it hits the cutoff.

Note: converting a float to an int tends to invoke trunc() or something
similar. trunc(14.999999999999) = 14. You need to decide whether maybe
you want round() instead. Or decide the actual cutoff yourself. I
sometimes fix this by adding 0.00001 to numbers before converting to
integer, to avoid these issues.

Floating points numbers are accurate but not precise.

Hope this helps,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2005-11-04 16:49:51 Re: somebody could explain this?
Previous Message Andrew Dunstan 2005-11-04 16:45:31 Re: somebody could explain this?