Re: [HACKERS] dollar quoting

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> I am a little concerned about adding the overhead of lex to psql. Right
> now, some folks have reported that lex/yacc take a considerable amount
> of processing time in the backend as part of a query, and adding that to
> psql just to do $$ seems questionable. Of course, we can alway test and
> see what the overhead shows.

That's not the question to ask --- the question is whether a flex lexer
will be faster or slower than the hand-made lexing code that psql is
currently using. Lexer generation is a well-understood art, and you
have to be pretty tense to beat out flex with hand-made code. It's the
same tradeoff as trying to write better assembly code than a compiler
does. Look at the lexing loop in psql/mainloop.c (that series of
if-tests starting at about line 250) and ask yourself if that's really
going to beat out the state-machine implementation flex uses --- which
looks to be about two table lookups per character, plus a switch()
executed at the end of every token. I'll bet on flex being faster.

The reason the lexer shows up in the backend is that it has to grovel
over every individual character of a query. For sufficiently large
query strings that's gonna take awhile no matter how you do it.

But in any case, the argument for moving to flex is not about
performance, it is about making the code more understandable and more
certain to agree with the backend lexer.

regards, tom lane