Re: Add regression tests for ROLE (USER)

On Mon, Jul 15, 2013 at 10:23 AM, Robins Tharakan <tharakan(at)gmail(dot)com> wrote:
> It still doesn't address the excessive (syntactical) checks tough. I am
> still unclear as to how to identify which checks to skip. (As in, although I
> have a personal preference of checking everything, my question probably
> wasn't clear in my previous email. I was just asking 'how' to know which
> checks to not perform.) Should a 'syntax error' in the message be considered
> as an unnecessary check? If so, its easier for me to identify which checks
> to skip, while creating future tests.

Yes, I think you can just leave out the syntax errors.

I simply don't understand how we can be getting any meaningful test
coverage out of those cases. I mean, if we want to check every bit of
syntax that could lead to a syntax error, we could probably come up
with a near-infinite number of test cases:

CREAT TABLE foo (x int);
CREATE TABL foo (x int);
CREATER TABLE foo (x int);
CREATE TABLES foo (x int);
CREATE CREATE TABLE foo (x int);
CREATE TABLE foo [x int);
CREATE TABLE foo (x int];
CREATE TABLE foo [x int];
CREATE TABLE (x int);
CREATE foo (x int);

...and on and on it goes. Once we start speculating that bison
doesn't actually produce a grammar that matches the productions
written in gram.y, there's no limit to what can go wrong, and no
amount of test coverage can be too large. In practice, the chances of
catching anything that way seem minute. If bison breaks in such a way
that all currently accepted grammatical constructs are still accepted
and work, but something that shouldn't have been accepted is, we'll
just have to find that out in some way other than our regression
tests. I think it's very unlikely that such a thing will happen, and
even if it does, I don't really see any reason to suppose that the
particular tests you've included here will be the ones that catch the
problem.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company