Re: numeric/decimal docs bug?

Bruce Momjian wrote:
> Jan Wieck wrote:
> > > The hard limit is certainly no more than 64K, since we store these
> > > numbers in half of an atttypmod. In practice I suspect the limit may
> > > be less; Jan would be more likely to remember...
> >
> > It is arbitrary of course. I don't recall completely, have to
> > dig into the code, but there might be some side effect when
> > mucking with it.
> >
> > The NUMERIC code increases the actual internal precision when
> > doing multiply and divide, what happens a gazillion times
> > when doing higher functions like trigonometry. I think there
> > was some connection between the max precision and how high
> > this internal precision can grow, so increasing the precision
> > might affect the computational performance of such higher
> > functions significantly.
>
> Oh, interesting, maybe we should just leave it alone.

As said, I have to look at the code. I'm pretty sure that it
currently will not use hundreds of digits internally if you
use only a few digits in your schema. So changing it isn't
that dangerous.

But who's going to write and run a regression test, ensuring
that the new high limit can really be supported. I didn't
even run the numeric_big test lately, which tests with 500
digits precision at least ... and therefore takes some time
(yawn). Increasing the number of digits used you first have
to have some other tool to generate the test data (I
originally used bc(1) with some scripts). Based on that we
still claim that our system deals correctly with up to 1,000
digits precision.

I don't like the idea of bumping up that number to some
higher nonsense, claiming we support 32K digits precision on
exact numeric, and noone ever tested if natural log really
returns it's result in that precision instead of a 30,000
digit precise approximation.

I missed some of the discussion, because I considered the
1,000 digits already beeing complete nonsense and dropped the
thread. So could someone please enlighten me what the real
reason for increasing our precision is? AFAIR it had
something to do with the docs. If it's just because the docs
and the code aren't in sync, I'd vote for changing the docs.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com