| From: | Scott Marlowe <smarlowe(at)g2switchworks(dot)com> |
|---|---|
| To: | pgsql-general(at)postgresql(dot)org |
| Subject: | numeric precision when raising one numeric to another. |
| Date: | 2005-05-18 21:17:49 |
| Message-ID: | 1116451069.31821.60.camel@state.g2switchworks.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-general pgsql-patches |
It appears from checking the output of exponentiation of one numeric to
another, the output is actually in floating point. Is this normal and /
or expected?
Now, given that
create table test2 (i1 numeric(20,0), i2 numeric(20,0));
insert into test values (123456789012345,123456789012345);
select i1*i2 from test2;
gives:
?column?
-------------------------------
15241578753238669120562399025
it seems odd that
create table test (i1 numeric(20,0), i2 numeric(20,0));
insert into test values (2,55);
select i1^i2 from test;
gives:
?column?
---------------------
3.6028797018964e+16
Now, I can get an exact answer if I'm willing to twiddle with breaking
the exponent down:
select (2^60)::numeric;
Gives:
numeric
---------------------
1152921504606850000
While, select (2^30)::numeric*(2^30)::numeric;
Gives:
?column?
---------------------
1152921504606846976
So, numeric can hold the value, but it looks like the exponent math is
converting it to float.
I'm not bothered too much by it, as I don't really work with numbers
that big. I was mainly wondering if this is kosher is all.
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tom Lane | 2005-05-18 21:42:28 | Re: numeric precision when raising one numeric to another. |
| Previous Message | Alvaro Herrera | 2005-05-18 21:10:54 | Re: Image storage questions |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tom Lane | 2005-05-18 21:42:28 | Re: numeric precision when raising one numeric to another. |
| Previous Message | Sergey Ten | 2005-05-18 20:42:57 | Re: patches for items from TODO list |