Re: Replacing plpgsql's lexer

On Fri, Apr 17, 2009 at 12:12:12PM -0400, Tom Lane wrote:
> I wrote:
> > I had earlier speculated semi-facetiously about ripping out the
> > plpgsql lexer altogether, but the more I think about it the less
> > silly the idea looks.
>
> This little project crashed and burned upon remembering that plpgsql
> invokes raw_parser() to syntax-check each chunk of SQL that it
> extracts. If plpgsql were using the main lexer, that would mean
> recursive use of the lexer --- and it's not re-entrant.
>
> We could think about making the main lexer re-entrant, but that
> would involve a bump in the minimum required flex version (I don't
> know when %option reentrant got added, but it's not in 2.5.4). And
> it definitely doesn't seem like something to be doing during beta.
>
> Getting rid of the requirement for recursion doesn't look palatable
> either. We don't want to delay the syntax check for reasons
> explained in check_sql_expr()'s comments; and that's not the only
> source of recursion anyway --- plpgsql_parse_datatype does it too,
> and there could be other places.
>
> So I think we are down to a choice of doing nothing for 8.4, or
> teaching the existing plpgsql lexer about
> standard_conforming_strings. Assuming the current proposal for U&
> literals holds up, it should not be necessary for plpgsql to know
> about those explicitly as long as it obeys
> standard_conforming_strings, so this might not be too horrid a
> project. I'll take a look at that next.

Speaking of standard_conforming_strings, I know it's late, but if we
make it a requirement now, a lot of problems just go away. Yes, it's
inconvenient, but we're making lots of big changes, so one more
shouldn't halt adoption.

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate