Re: Parser Hook

On Mon, Mar 15, 2021 at 11:48:58AM -0400, Jim Mlodgenski wrote:
>
> Going deeper on this, I created another POC as an example. Yes, having a
> hook at the top of the parser does mean an extension needs to copy the
> existing grammar and modify it. Without a total redesign of how the grammar
> is handled, I'm not seeing how else this could be accomplished. The example
> I have is adding a CREATE JOB command that a scheduler may use. The amount
> of effort needed for an extension maintainer doesn't appear to be that
> onerous. Its not ideal having to copy and patch gram.y, but certainly
> doable for someone wanting to extend the parser.

AFAIK nothing in bison prevents you from silently ignoring unhandled grammar
rather than erroring out. So you could have a parser hook called first, and
if no valid command was recognized fall back on the original parser. I'm not
saying that it's a good idea or will be performant (although the added grammar
will likely be very small, so it may not be that bad), but you could definitely
avoid the need to duplicate the whole grammar in each and every extension, and
allow multiple extensions extending the grammar.

That won't reduce the difficulty of producing a correct parse tree if you want
to implement some syntactic sugar for already handled DML though.

> However we would want to modify the parser to allow it to be more plugable
> in the future, we would very likely need to have a hook at the top of the
> parser to intiailize things like keywords. Having a hook at the top of the
> parser along with the existing ProcessUtility_hook allows extension to add
> their own utility commands if they wish. I would image that covers many
> existing use cases for extensions today.

What happens if multiple extensions want to add their own new grammar? There
will at least be possible conflicts with the additional node tags.

Also, I'm not sure that many extensions would really benefit from custom
utility command, as you can already do pretty much anything you want using SQL
functions. For instance it would be nice for hypopg to be able to support

CREATE HYPOTHETICAL INDEX ...

rather than

SELECT hypopg_create_index('CREATE INDEX...')

But really the only benefit would be autocompletion, which still wouldn't be
possible as psql autocompletion won't be extended. And even if it somehow was,
I wouldn't expect all psql clients to be setup as needed.