Re: Re: BUG #12885: The result of casting a double to an integer depends on the database version

From: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
To: Pedro Gimeno <pgsql-004(at)personal(dot)formauri(dot)es>
Cc: Andrew Gierth <andrew(at)tao11(dot)riddles(dot)org(dot)uk>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, rschaaf(at)commoninf(dot)com, pgsql-bugs(at)postgresql(dot)org
Subject: Re: Re: BUG #12885: The result of casting a double to an integer depends on the database version
Date: 2015-03-27 20:55:13
Message-ID: CAB7nPqTExJZDPdzbs3hsTqx845T0epqH9GXYhA1W7uopyExtAQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

On Sat, Mar 28, 2015 at 2:53 AM, Pedro Gimeno
<pgsql-004(at)personal(dot)formauri(dot)es> wrote:
> Michael Paquier wrote, On 2015-03-27 13:32:
>
>> Attached is a patch that adds nearest-to-even rounding for numeric
>> when dscale = 0. This gives the following results with the previous
>> query:
>
> That produces some quite surprising results:
>
> =# select '2.5'::numeric(9,0)::numeric(9,1),
> '2.25'::numeric(9,1)::numeric(9,2),
> '2.225'::numeric(9,2)::numeric(9,3),
> '2.2225'::numeric(9,3)::numeric(9,4),
> '2.22225'::numeric(9,4)::numeric(9,5);
> numeric | numeric | numeric | numeric | numeric
> ---------+---------+---------+---------+---------
> 2.0 | 2.30 | 2.230 | 2.2230 | 2.22220
> (1 row)
>
> That rounding is inconsistent: since DEC_DIGITS is 4 by default, numbers
> with a number of digits which is a multiple of 4 + 1 will be rounded to
> nearest or even, and numbers with a different number of digits will be
> rounded up on a tie. And I have not tested, but apparently when
> DEC_DIGITS == 1 (array elements are single digits rather than packing
> several) it always rounds down on a tie.

Yeah, I forgot a check on dscale = 0 when deciding to carry to the
upper digit or not.
--
Michael

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message Michael Paquier 2015-03-27 21:31:59 Re: Re: BUG #12885: The result of casting a double to an integer depends on the database version
Previous Message Jon Jensen 2015-03-27 20:41:08 Re: pg_get_constraintdef() doesn't always give an equal constraint