Re: Incorrect rounding of double values at max precision

>>>>> "Tom" == Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

>> Perhaps it would make more sense for the float8 to numeric cast to
>> look at the requested typmod and use that for the conversion?

Tom> As things stand right now, float8_numeric has no idea what the
Tom> target typmod is; any typmod-driven rounding happens in a separate
Tom> function call afterwards. I don't recall whether the parser's
Tom> casting infrastructure could support merging those steps,

As far as I can tell it does; it looks at whether the cast function
takes a typmod parameter, and if it does, it passes the typmod,
otherwise it generates a separate typmod coercion and stacks that on top
of the cast proper.

So changing the function declaration to include a typmod parameter, and
using it, should just work... but I've not tested it yet.

Tom> and I'm not sure it matters in most cases. Commonly, we don't have
Tom> a target typmod. (Still, if we do, having two separate rounding
Tom> steps isn't nice.)

So we'd still need to decide what to do in the no-typmod case.

--
Andrew (irc:RhodiumToad)