Re: WIP Incremental JSON Parser

On 2024-03-26 Tu 17:52, Jacob Champion wrote:
> On Mon, Mar 25, 2024 at 3:14 PM Jacob Champion
> <jacob(dot)champion(at)enterprisedb(dot)com> wrote:
>> - add Assert calls in impossible error cases [2]
> To expand on this one, I think these parts of the code (marked with
> `<- here`) are impossible to reach:
>
>> switch (top)
>> {
>> case JSON_TOKEN_STRING:
>> if (next_prediction(pstack) == JSON_TOKEN_COLON)
>> ctx = JSON_PARSE_STRING;
>> else
>> ctx = JSON_PARSE_VALUE; <- here
>> break;
>> case JSON_TOKEN_NUMBER: <- here
>> case JSON_TOKEN_TRUE: <- here
>> case JSON_TOKEN_FALSE: <- here
>> case JSON_TOKEN_NULL: <- here
>> case JSON_TOKEN_ARRAY_START: <- here
>> case JSON_TOKEN_OBJECT_START: <- here
>> ctx = JSON_PARSE_VALUE;
>> break;
>> case JSON_TOKEN_ARRAY_END: <- here
>> ctx = JSON_PARSE_ARRAY_NEXT;
>> break;
>> case JSON_TOKEN_OBJECT_END: <- here
>> ctx = JSON_PARSE_OBJECT_NEXT;
>> break;
>> case JSON_TOKEN_COMMA: <- here
>> if (next_prediction(pstack) == JSON_TOKEN_STRING)
>> ctx = JSON_PARSE_OBJECT_NEXT;
>> else
>> ctx = JSON_PARSE_ARRAY_NEXT;
>> break;
> Since none of these cases are non-terminals, the only way to get to
> this part of the code is if (top != tok). But inspecting the
> productions and transitions that can put these tokens on the stack, it
> looks like the only way for them to be at the top of the stack to
> begin with is if (tok == top). (Otherwise, we would have chosen a
> different production, or else errored out on a non-terminal.)
>
> This case is possible...
>
>> case JSON_TOKEN_STRING:
>> if (next_prediction(pstack) == JSON_TOKEN_COLON)
>> ctx = JSON_PARSE_STRING;
> ...if we're in the middle of JSON_PROD_[MORE_]KEY_PAIRS. But the
> corresponding else case is not, because if we're in the middle of a
> _KEY_PAIRS production, the next_prediction() _must_ be
> JSON_TOKEN_COLON.
>
> Do you agree, or am I missing a way to get there?
>

One good way of testing would be to add the Asserts, build with
-DFORCE_JSON_PSTACK, and run the standard regression suite, which has a
fairly comprehensive set of JSON errors. I'll play with that.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com