Re: [PATCH] Generalized JSON output functions

On 07/13/2015 05:41 AM, Shulgin, Oleksandr wrote:
> On Mon, Jul 13, 2015 at 9:44 AM, Pavel Stehule
> <pavel(dot)stehule(at)gmail(dot)com <mailto:pavel(dot)stehule(at)gmail(dot)com>> wrote:
>
>
> To reiterate: for my problem, that is escaping numerics that
> can potentially overflow[1] under ECMAScript standard, I want
> to be able to override the code that outputs the numeric
> converted to string. There is no way in current
> implementation to do that *at all*, short of copying all the
> code involved in producing JSON output and changing it at
> certain points. One could try re-parsing JSON instead, but
> that doesn't actually solve the issue, because type
> information is lost forever at that point.
>
> The whitespace unification was a mere side-effect of the
> original effort on this patch.
>
>
> The dynamic type change is some what I would not to do in
> database, really :)
>
> If you afraid about overflow, then convert numeric to string
> immediately - in this case, the client have to support both
> variant - so immediate cast should not be a problem.
>
>
> Yeah, but how would you do that in context of a logical replication
> decoding plugin? I've tried a number of tricks for that, including,
> but not limited to registering phony types to wrap numeric type and
> replacing the OID of numeric with this custom type OID in TupleDesc,
> but then again one has to register that as known record type, etc.
>
> Anyway this check on max number should be implemented in our
> JSON(b) out functions (as warning?).
>
>
> Not really, since this is a problem of ECMAScript standard, not JSON
> spec. For example, Python module for handling JSON doesn't suffer
> from this overflow problem,
>
> The thing is, we cannot know which clients are going to consume the
> stream of decoded events, and if it's some implementation of
> JavaScript, it can suffer silent data corruption if we don't guard
> against that in the logical decoding plugin.
>
> Hope that makes it clear. :-)
>
>

Yes, but I think the plugin is the right place to do it. What is more,
this won't actually prevent you completely from producing non-ECMAScript
compliant JSON, since json or jsonb values containing offending numerics
won't be caught, AIUI. But a fairly simple to write function that
reparsed and fixed the JSON inside the decoder would work.

cheers

andrew