From: | Scott Marlowe <smarlowe(at)g2switchworks(dot)com> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | numeric precision when raising one numeric to another. |
Date: | 2005-05-18 21:17:49 |
Message-ID: | 1116451069.31821.60.camel@state.g2switchworks.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general pgsql-patches |
It appears from checking the output of exponentiation of one numeric to
another, the output is actually in floating point. Is this normal and /
or expected?
Now, given that
create table test2 (i1 numeric(20,0), i2 numeric(20,0));
insert into test values (123456789012345,123456789012345);
select i1*i2 from test2;
gives:
?column?
-------------------------------
15241578753238669120562399025
it seems odd that
create table test (i1 numeric(20,0), i2 numeric(20,0));
insert into test values (2,55);
select i1^i2 from test;
gives:
?column?
---------------------
3.6028797018964e+16
Now, I can get an exact answer if I'm willing to twiddle with breaking
the exponent down:
select (2^60)::numeric;
Gives:
numeric
---------------------
1152921504606850000
While, select (2^30)::numeric*(2^30)::numeric;
Gives:
?column?
---------------------
1152921504606846976
So, numeric can hold the value, but it looks like the exponent math is
converting it to float.
I'm not bothered too much by it, as I don't really work with numbers
that big. I was mainly wondering if this is kosher is all.
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2005-05-18 21:42:28 | Re: numeric precision when raising one numeric to another. |
Previous Message | Alvaro Herrera | 2005-05-18 21:10:54 | Re: Image storage questions |
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2005-05-18 21:42:28 | Re: numeric precision when raising one numeric to another. |
Previous Message | Sergey Ten | 2005-05-18 20:42:57 | Re: patches for items from TODO list |