Re: numeric precision when raising one numeric to

From: "Jim C(dot) Nasby" <decibel(at)decibel(dot)org>
To: Stephan Szabo <sszabo(at)megazone(dot)bigpanda(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Scott Marlowe <smarlowe(at)g2switchworks(dot)com>, "John D(dot) Burger" <john(at)mitre(dot)org>, pgsql-general(at)postgresql(dot)org
Subject: Re: numeric precision when raising one numeric to
Date: 2005-05-20 16:42:44
Message-ID: 20050520164244.GX44623@decibel.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

That's because numerics default to 16 or something similar. If you want
more precision just explicitly cast it:

decibel=# select power(0.1::numeric(20,20),17);
0.00000000000000001000

On Fri, May 20, 2005 at 09:30:16AM -0700, Stephan Szabo wrote:
>
> On Fri, 20 May 2005, Tom Lane wrote:
>
> > Has anyone bothered to actually look into the code?
> >
> > regression=# select power(2::numeric,1000);
> > power
> > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > 10715086071862673209484250490600018105614048117055336074437503883703510511249361224931983788156958581275946729175531468251871452856923140435984577574698574803934567774824230985421074605062371141877954182153046474983581941267398767559165543946077062914571196477686542167660429831652624386837205668069376.0000000000000000
> > (1 row)
> >
> > AFAICT the only thing missing is a pg_operator entry linked to the
> > function.
>
> It appears fairly limited however given that you rapidly run into the
> numeric maximum length for exp.
>
> It also doesn't seem to work terribly well:
>
> sszabo=# select power(0.1::numeric, 15);
> power
> --------------------
> 0.0000000000000010
> (1 row)
>
> sszabo=# select power(0.1::numeric, 16);
> power
> --------------------
> 0.0000000000000001
> (1 row)
>
> sszabo=# select power(0.1::numeric, 17);
> power
> --------------------
> 0.0000000000000000
> (1 row)
>
> sszabo=# select power(0.1::numeric, 17)*100;
> ?column?
> --------------------
> 0.0000000000000000
> (1 row)
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo(at)postgresql(dot)org
>

--
Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2005-05-20 16:43:49 Re: numeric precision when raising one numeric to
Previous Message Scott Marlowe 2005-05-20 16:32:33 Re: numeric precision when raising one numeric to