| From: | Christophe Pettus <xof(at)thebuild(dot)com> |
|---|---|
| To: | Raphaël Berbain <raphael(dot)berbain(at)gmail(dot)com> |
| Cc: | pgsql-general(at)lists(dot)postgresql(dot)org |
| Subject: | Re: width_bucket issue |
| Date: | 2018-07-24 20:09:29 |
| Message-ID: | D1114C6A-9A61-468C-817E-86D467198569@thebuild.com |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-general |
> On Jul 24, 2018, at 13:02, Raphaël Berbain <raphael(dot)berbain(at)gmail(dot)com> wrote:
> I'd expect b1 = b2 = 2. What am I missing?
The problem appears to be due to rounding during the intermediate calculations. In compute_bucket() in numeric.c:
div_var(&operand_var, &bound1_var, result_var,
select_div_scale(&operand_var, &bound1_var), true);
... produces 0.99999999999999999999 for that particular value, instead of 1, and the subsequent +1 and FLOOR() result in 1 instead of 2.
--
-- Christophe Pettus
xof(at)thebuild(dot)com
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Jeff Janes | 2018-07-25 02:13:31 | Re: Dealing with latency to replication slave; what to do? |
| Previous Message | Rory Falloon | 2018-07-24 20:08:38 | Re: Dealing with latency to replication slave; what to do? |