Re: WIP: rewrite numeric division

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Gregory Stark <stark(at)enterprisedb(dot)com>
Cc: pgsql-patches(at)postgresql(dot)org
Subject: Re: WIP: rewrite numeric division
Date: 2007-06-19 14:28:29
Message-ID: 27542.1182263309@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-patches

Gregory Stark <stark(at)enterprisedb(dot)com> writes:
> "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:
>> Yeah, my proposed patch is schoolbook division. I don't think I'd trust
>> Newton's method to give anything but a close approximation, which is
>> what we have in HEAD already.

> Well unless we start storing rational numbers it'll always be a limited to a
> finite number of decimals. The key is whether we can guarantee a lower bound
> on the number of accurate decimal places. As long as we can then we can keep
> going until the decimal places we're going to return are all accurate.

This is nonsense. Consider an approximate division that gives

...123.999999999999999999...

You can keep generating 9's forever, but you'll never know whether the
accurate result of trunc() should be 123 or 124. Unless you use a
method that gives you trustworthy digits to start with.

> It looks like multiplication can also generate incorrect results.

It can, but only if told to produce fewer digits than would be in the
exact result, so I'm not worried about that.

regards, tom lane

In response to

Browse pgsql-patches by date

  From Date Subject
Next Message Tom Lane 2007-06-19 17:28:55 Re: [HACKERS] 'Waiting on lock'
Previous Message Magnus Hagander 2007-06-19 13:04:11 Re: Preliminary GSSAPI Patches