Re: WIP Incremental JSON Parser

On Wed, Feb 21, 2024 at 6:50 AM Jacob Champion
<jacob(dot)champion(at)enterprisedb(dot)com> wrote:
> On Tue, Feb 20, 2024 at 9:32 PM Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
> > *sigh* That's weird. I wonder why you can reproduce it and I can't. Can
> > you give me details of the build? OS, compiler, path to source, build
> > setup etc.? Anything that might be remotely relevant.

This construction seems suspect, in json_lex_number():

> if (lex->incremental && !lex->inc_state->is_last_chunk &&
> len >= lex->input_length)
> {
> appendStringInfoString(&lex->inc_state->partial_token,
> lex->token_start);
> return JSON_INCOMPLETE;
> }

appendStringInfoString() isn't respecting the end of the chunk: if
there's extra data after the chunk boundary (as
AppendIncrementalManifestData() does) then all of that will be stuck
onto the end of the partial_token.

I'm about to context-switch off of this for the day, but I can work on
a patch tomorrow if that'd be helpful. It looks like this is not the
only call to appendStringInfoString().

--Jacob