Re: numeric precision when raising one numeric to another.

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Scott Marlowe <smarlowe(at)g2switchworks(dot)com>
Cc: "Jim C(dot) Nasby" <decibel(at)decibel(dot)org>, pgsql-general(at)postgresql(dot)org
Subject: Re: numeric precision when raising one numeric to another.
Date: 2005-05-20 18:26:08
Message-ID: 26839.1116613568@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-patches

Scott Marlowe <smarlowe(at)g2switchworks(dot)com> writes:
> Are you saying that the exponent operator will return inexact results?

For a fractional exponent, it generally has to, because there is no
finite exact result.

> If you're quoting the 92 spec, it seems to say that multiplication
> precision is also implementation specific.

You're misreading it: the scale part is what's important. Precision
means the implementation gets to set a limit on the total number of
digits it will store. The scale rules effectively say that for add/sub/mul
you must either deliver an exact result or report overflow. But that
requirement is not placed on division, and (by implication) not on other
operations that are incapable of delivering exact results every time.

numeric_power can in theory deliver an exact answer when the exponent is
a positive integer. Division can deliver an exact answer in some cases
too --- but the spec doesn't say it must do so when possible. So I
would say that there is no spec requirement for special behavior for
integral exponents.

We could try to deliver an exact answer for an integral exponent by
selecting output scale = input scale times exponent. But that doesn't
work for any but very small exponents --- as the exponent gets bigger
you really have to drop fractional precision, or you're going to hit
overflow, which is not an improvement. (We do set a limit on total
number of digits...) So it's a question of tradeoffs, not black and
white.

regards, tom lane

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Scott Marlowe 2005-05-20 18:31:28 Re: numeric precision when raising one numeric to
Previous Message jjeffman 2005-05-20 18:01:34 Re: table synonyms

Browse pgsql-patches by date

  From Date Subject
Next Message Andrew Dunstan 2005-05-20 21:33:46 Re: patches for items from TODO list
Previous Message Scott Marlowe 2005-05-20 17:29:10 Re: numeric precision when raising one numeric to