Re: Parser Hook

On Mon, Feb 22, 2021 at 3:52 PM Andres Freund <andres(at)anarazel(dot)de> wrote:

> Hi,
>
> On 2021-02-22 11:20:54 -0500, Jim Mlodgenski wrote:
> > As Jan mentioned in his thread about a pluggable wire protocol [0], AWS
> is
> > working on a set of extensions for Babelfish. The intention is to not
> > necessarily have it as a single monolithic extension, but be possible for
> > people to use pieces of it as they need when they are migrating to
> > PostgreSQL. Some may just need the functions or data types. Others may
> need
> > the stored procedure language. Many times when enterprises are migrating
> > databases, they have satellite applications that they may not be able to
> > change or they are on a different schedules than the main application so
> > the database still needs to support some of the old syntax. A common need
> > in these situations is the parser.
> >
> > Attached is a patch to place a hook at the top of the parser to allow
> for a
> > pluggable parser. It is modeled after the planner_hook [1]. To test the
> > hook, I have also attached a simple proof of concept that wraps the
> parser
> > in a TRY/CATCH block to catch any parse errors. That could potentially
> help
> > a class of users who are sensitive to parse errors ending up in the logs
> > and leaking PII data or passwords.
>
> I don't think these are really comparable. In case of the planner hook
> you can reuse the normal planner pieces, and just deal with the one part
> you need to extend. But we have pretty much no infrastructure to use the
> parser in a piecemeal fashion (there's a tiny bit for plpgsql).
>
> Which in turn means that to effectively use the proposed hook to
> *extend* what postgres accepts, you need to copy the existing parser,
> and hack in your extensions. Which in turn invariably will lead to
> complaints about parser changes / breakages the community will get
> complaints about in minor releases etc.
>

Going deeper on this, I created another POC as an example. Yes, having a
hook at the top of the parser does mean an extension needs to copy the
existing grammar and modify it. Without a total redesign of how the grammar
is handled, I'm not seeing how else this could be accomplished. The example
I have is adding a CREATE JOB command that a scheduler may use. The amount
of effort needed for an extension maintainer doesn't appear to be that
onerous. Its not ideal having to copy and patch gram.y, but certainly
doable for someone wanting to extend the parser. I also extended the patch
to add another hook in parse_expr.c to see what we would need to add
another keyword and have it call a function like SYSDATE. That appears to
be a lot of work to get all of the potentail hook points that an extension
may want to add and there may not be that many usecases worth the effort.

> I think the cost incurred for providing a hook that only allows
> extensions to replace the parser with a modified copy of ours will be
> higher than the gain. Note that I'm not saying that I'm against
> extending the parser, or hooks - just that I don't think just adding the
> hook is a step worth doing on its own.
>
>
However we would want to modify the parser to allow it to be more plugable
in the future, we would very likely need to have a hook at the top of the
parser to intiailize things like keywords. Having a hook at the top of the
parser along with the existing ProcessUtility_hook allows extension to add
their own utility commands if they wish. I would image that covers many
existing use cases for extensions today.

> Imo a functional approach would really need to do the work to allow to
> extend & reuse the parser in a piecemeal fashion and *then* add a hook.
>
> Greetings,
>
> Andres Freund
>