Re: Consider \v to the list of whitespace characters in the parser

On Mon, Jul 03, 2023 at 12:17:10PM +0200, Peter Eisentraut wrote:
> SQL has "whitespace", which includes any Unicode character with the
> White_Space property (which includes \v), and <newline>, which is
> implementation-defined.
>
> So nothing there speaks against treating \v as a (white)space character in
> the SQL scanner.

Okay, thanks for confirming.

> In scan.l, you might want to ponder horiz_space: Even though \v is clearly
> not "horizontal space", horiz_space already includes \f, which is also not
> horizontal IMO. I think horiz_space is really all space characters except
> newline characters. Maybe this should be rephrased.

And a few lines above, there is a comment from 2000 (3cfdd8f)
pondering if \f should be handled as a newline, which is kind of
incorrect anyway?

FWIW, I agree that horiz_space is confusing in this context because it
does not completely reflect the reality, and \v is not that so adding
it to the existing list felt wrong to me. Form feed is also not a
newline, from what I understand.. From what the parser tells, there
are two things we want to track to handle comments:
- All space characters, which would be \t\n\r\f\v.
- All space characters that are not newlines, \t\f\v.

I don't really have a better idea this morning than using the
following terms in the parser, changing the surroundings with similar
terms:
-space [ \t\n\r\f]
-horiz_space [ \t\f]
+space [ \t\n\r\f\v]
+non_newline_space [ \t\f\v]

Perhaps somebody has a better idea of split?
--
Michael