Re: Hook for extensible parsing.

On Wed, Jul 7, 2021 at 8:45 PM Jim Mlodgenski <jimmy76(at)gmail(dot)com> wrote:
>
> The test module is very useful to show how to use the hook but it isn't
> very useful to the general user like most other things in contrib. It probably
> fits better in src/test/modules

I agree that it's not useful at all to eventually have it as a
contrib, but it's somewhat convenient at this stage to be able to
easily test the hook, possibly with different behavior.

But as I said, if there's an agreement on the approach and the
implementation, I don't think that it would make sense to keep it even
in the src/test/modules. A full bison parser, even with a limited
grammar, will have about 99% of noise when it comes to demonstrate how
the hook is supposed to work, which basically is having a "single
query" parser or a "full input string" parser. I'm not even convinced
that flex/bison will be the preferred choice for someone who wants to
implement a custom parser.

I tried to add really thorough comments in the various parts of the
patch to make it clear how to do that and how the system will react
depending on what a hook does. I also added some protection to catch
inconsistent hook implementation. I think that's the best way to help
external parser authors to implement what they want, and I'll be happy
to improve the comments if necessary. But if eventually people would
like to have a real parser in the tree, for testing or guidance, I
will of course take care of doing the required changes and moving the
demo parser in src/test/modules.