Re: Bug in numeric multiplication

From: Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Bug in numeric multiplication
Date: 2015-09-21 15:21:10
Message-ID: CAEZATCWys0N9PVQUA27iUjw0rDauAm35nwGHJ7PKFacxtXnFiQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 21 September 2015 at 16:09, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> I wrote:
>> Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> writes:
>>> The problem then arises in the final carry propagation pass. During
>>> this phase of the computation, the carry from one digit (which can be
>>> a shade under INT_MAX / NBASE) is added to the next digit, and that's
>>> where the overflow happens.
>
>> Nice catch! I think the comment could use a little more work, but I'll
>> adjust it and push.
>
> After trying to rework the comment to explain what maxdig really meant
> after your changes, I came to the conclusion that it'd be better to do
> it as per attached. Does this look sane to you?
>

Yes that looks better. It's still the same amount of extra headroom
(21), but I think it's clearer your way.

Regards,
Dean

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dmitry Dolgov 2015-09-21 16:13:05 Re: jsonb_set array append hack?
Previous Message Tom Lane 2015-09-21 15:09:14 Re: Bug in numeric multiplication