| From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
|---|---|
| To: | Nuno Sousa <nunofgs(at)gmail(dot)com> |
| Cc: | pgsql-novice(at)lists(dot)postgresql(dot)org |
| Subject: | Re: Numeric Scale Differences |
| Date: | 2021-10-21 14:06:04 |
| Message-ID: | 2404795.1634825164@sss.pgh.pa.us |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-novice |
Nuno Sousa <nunofgs(at)gmail(dot)com> writes:
> I've run into an interesting issue when trying to do numeric math. I
> expected the following to return the same number of decimal digits in the
> fractional part:
> select scale(1/0.6::numeric), scale(1/6::numeric);
Um ... why did you expect that? The inputs to the divisions have
different scales:
# select scale(0.6::numeric), scale(6::numeric);
scale | scale
-------+-------
1 | 0
(1 row)
so I find it unsurprising that the outputs do too. Now the fact
that the output scales differ by 4 not 1 is indeed an implementation
artifact. It stems from the numeric type working with base-10000
digits, so that scales that are multiples of 4 are most efficient,
hence division will always choose such an output scale.
regards, tom lane
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Nuno Sousa | 2021-10-21 15:26:26 | Re: Numeric Scale Differences |
| Previous Message | Nuno Sousa | 2021-10-21 13:47:48 | Numeric Scale Differences |