Re: Arbitrary precision modulo operation

Alvaro Herrera wrote:

>On Mon, Apr 26, 2004 at 12:48:45PM -0700, Dann Corbit wrote:
>
>
>>Maple output:
>>y := 123456789012345678901234567890 mod 123;
>> y := 117
>>
>>
>
>PgSQL 7.3.6 gives the right answer (117), 7.4 gets it wrong (-6). Most
>likely a bug was introduced when NUMERIC was rewritten. Strange it
>hasn't been noticed before.
>
>
>
mod(x, y) is computed as x - trunc(x / y) * y in the mod_var() function
(I think).

However, it appears that the division operator itself is rounding up,
such that the trunc() function (which ought to round down) does no good
as a round up has already occurred.

Thus, the value of (x / y) is 1 too large, and so x % y is actually
giving you (x % y) - y, a negative number. I tried looking at how the
division actually works, but it is over my head at least for the 30
minute perusal.

Regards,
Paul Tillotson

-----------------------------------------------------------------------

[paul(at)pjt4 paul]$ bc
bc 1.06
Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
111111111111111111 / 6
18518518518518518

[paul(at)pjt4 bin]$ ./psql -U postgres template1
Welcome to psql 7.4.2, the PostgreSQL interactive terminal.

Type: \copyright for distribution terms
\h for help with SQL commands
\? for help on internal slash commands
\g or terminate with semicolon to execute query
\q to quit

template1=# select 111111111111111111::numeric / 6;
?column?
-------------------
18518518518518519
(1 row)

template1=# select 111111111111111111 / 6;
?column?
-------------------
18518518518518518
(1 row)

template1=# select version();

version

---------------------------------------------------------------------------------------------------------
PostgreSQL 7.4.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.3.2
20031022 (Red Hat Linux 3.3.2-1)
(1 row)