numeric precision when raising one numeric to another.

From: Scott Marlowe <smarlowe(at)g2switchworks(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: numeric precision when raising one numeric to another.
Date: 2005-05-18 21:17:49
Message-ID: 1116451069.31821.60.camel@state.g2switchworks.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-patches

It appears from checking the output of exponentiation of one numeric to
another, the output is actually in floating point. Is this normal and /
or expected?

Now, given that

create table test2 (i1 numeric(20,0), i2 numeric(20,0));
insert into test values (123456789012345,123456789012345);
select i1*i2 from test2;
gives:
?column?
-------------------------------
15241578753238669120562399025

it seems odd that

create table test (i1 numeric(20,0), i2 numeric(20,0));
insert into test values (2,55);
select i1^i2 from test;
gives:
?column?
---------------------
3.6028797018964e+16

Now, I can get an exact answer if I'm willing to twiddle with breaking
the exponent down:

select (2^60)::numeric;

Gives:
numeric
---------------------
1152921504606850000

While, select (2^30)::numeric*(2^30)::numeric;

Gives:
?column?
---------------------
1152921504606846976

So, numeric can hold the value, but it looks like the exponent math is
converting it to float.

I'm not bothered too much by it, as I don't really work with numbers
that big. I was mainly wondering if this is kosher is all.

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2005-05-18 21:42:28 Re: numeric precision when raising one numeric to another.
Previous Message Alvaro Herrera 2005-05-18 21:10:54 Re: Image storage questions

Browse pgsql-patches by date

  From Date Subject
Next Message Tom Lane 2005-05-18 21:42:28 Re: numeric precision when raising one numeric to another.
Previous Message Sergey Ten 2005-05-18 20:42:57 Re: patches for items from TODO list