| From: | "Jim C(dot) Nasby" <decibel(at)decibel(dot)org> |
|---|---|
| To: | Stephan Szabo <sszabo(at)megazone(dot)bigpanda(dot)com> |
| Cc: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Scott Marlowe <smarlowe(at)g2switchworks(dot)com>, "John D(dot) Burger" <john(at)mitre(dot)org>, pgsql-general(at)postgresql(dot)org |
| Subject: | Re: numeric precision when raising one numeric to |
| Date: | 2005-05-20 16:42:44 |
| Message-ID: | 20050520164244.GX44623@decibel.org |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-general |
That's because numerics default to 16 or something similar. If you want
more precision just explicitly cast it:
decibel=# select power(0.1::numeric(20,20),17);
0.00000000000000001000
On Fri, May 20, 2005 at 09:30:16AM -0700, Stephan Szabo wrote:
>
> On Fri, 20 May 2005, Tom Lane wrote:
>
> > Has anyone bothered to actually look into the code?
> >
> > regression=# select power(2::numeric,1000);
> > power
> > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > 10715086071862673209484250490600018105614048117055336074437503883703510511249361224931983788156958581275946729175531468251871452856923140435984577574698574803934567774824230985421074605062371141877954182153046474983581941267398767559165543946077062914571196477686542167660429831652624386837205668069376.0000000000000000
> > (1 row)
> >
> > AFAICT the only thing missing is a pg_operator entry linked to the
> > function.
>
> It appears fairly limited however given that you rapidly run into the
> numeric maximum length for exp.
>
> It also doesn't seem to work terribly well:
>
> sszabo=# select power(0.1::numeric, 15);
> power
> --------------------
> 0.0000000000000010
> (1 row)
>
> sszabo=# select power(0.1::numeric, 16);
> power
> --------------------
> 0.0000000000000001
> (1 row)
>
> sszabo=# select power(0.1::numeric, 17);
> power
> --------------------
> 0.0000000000000000
> (1 row)
>
> sszabo=# select power(0.1::numeric, 17)*100;
> ?column?
> --------------------
> 0.0000000000000000
> (1 row)
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo(at)postgresql(dot)org
>
--
Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
Give your computer some brain candy! www.distributed.net Team #1828
Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tom Lane | 2005-05-20 16:43:49 | Re: numeric precision when raising one numeric to |
| Previous Message | Scott Marlowe | 2005-05-20 16:32:33 | Re: numeric precision when raising one numeric to |