Re: Optimize numeric multiplication for one and two base-NBASE digit multiplicands.

From: "Joel Jacobson" <joel(at)compiler(dot)org>
To: "Dean Rasheed" <dean(dot)a(dot)rasheed(at)gmail(dot)com>
Cc: Dagfinn Ilmari Mannsåker <ilmari(at)ilmari(dot)org>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Optimize numeric multiplication for one and two base-NBASE digit multiplicands.
Date: 2024-07-02 19:55:19
Message-ID: 27d42b25-f328-4231-9db0-3811dc77c8cf@app.fastmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Jul 2, 2024, at 20:53, Joel Jacobson wrote:
> Trying to wrap my head around what could cause this.
>
> It's rounding down instead of up, and these cases all end with decimal .500XXXX.

Interesting, I actually think there is a bug in the normal mul_var() code.
Found a case that rounds down, when it should round up:

Calling mul_var() with:
var1=51.2945442386599
var2=0.828548712212
rscale=0

returns 42, but I think it should return 43,
since 51.2945442386599*0.828548712212=42.5000285724431241296446988

But maybe this is expected and OK, having to do with MUL_GUARD_DIGITS?

/Joel

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2024-07-02 20:00:48 Re: call for applications: mentoring program for code contributors
Previous Message Robert Haas 2024-07-02 19:54:17 Re: On disable_cost