Re: MySQL's "crashme" (was Re: Performance)

"Matthias Urlichs" <smurf(at)noris(dot)net> writes:
> Tom Lane:
>> I would not like to see us labeled "crashme unsafe" merely because
>> someone is too impatient to let the test run to conclusion.

> There's not much anybody can do about a backend which dies because of a
> "hard" out-of-memory error which the OS only notices when all it can do
> is segfault the client.

I'm just saying that it's unfair to downrate us when the problem is
demonstrably in crashme itself and not in Postgres.

>>>> alter_drop_col=no # Alter table drop column
>>
>> While our ALTER TABLE support is certainly pretty weak, it should be
>> noted that this test will continue to fail even when we have ALTER TABLE
>> DROP COLUMN, because crashme is testing for a non-SQL-compliant syntax.
>>
> You mean because the COLUMN keyword is missing? Added.

No, the COLUMN keyword is optional according to the SQL92 specification:

<alter table statement> ::=
ALTER TABLE <table name> <alter table action>

<alter table action> ::=
<add column definition>
| <alter column definition>
| <drop column definition>
| <add table constraint definition>
| <drop table constraint definition>

<drop column definition> ::=
DROP [ COLUMN ] <column name> <drop behavior>

<drop behavior> ::= CASCADE | RESTRICT

What is *not* optional is a <drop behavior> keyword. Although we don't
yet implement DROP COLUMN, our parser already has this statement in it
--- and it follows the SQL92 grammar.

>>>> comment_#=no # # as comment
>>
>> It'd be helpful to the reader if they indicated which two of these
>> conventions are SQL-compliant ... of course, that might expose the
>> fact that MySQL isn't ...
>>
> Are there any problems with using '#' as a comment character, given that
> MySQL doesn't support user-defined operators?

Only in that your scripts don't port to spec-compliant DBMSes ...

>>>> Some of the tests you are about to execute require a lot of memory.
>>>> Your tests _will_ adversely affect system performance. Either this
>>>> crash-me test program, or the actual database back-end, _will_ die with
>>>> an out-of-memory error. So might any other program on your system if it
>>>> requests more memory at the wrong time.

> Good enough?

No, pretty misleading I'd say. Since the crashme script does have a
limit on max_buffer_size, it *will* run to completion if run on a
machine with a sufficiently large per-process memory limit (and enough
swap of course). I may just be old-fashioned in running with a
not-so-large memory limit. It'd probably be more helpful if you
document the behavior seen when crashme runs out of memory (for me,
the script just stops cold with no notice) and what to do about it
(reduce max_buffer_size until it runs to completion).

>>>> date_zero=no # Supports 0000-00-00 dates
>> Another test that only MySQL "passes".
> ... and SOLID.

Still doesn't mean it's a good idea ;-)

>>>> except=no # except
>> Unsurprisingly, hardly anyone else accepts it either.
> SOLID again.

It'd be appropriate to divide this into two tests, or at least relabel
it.

>>>> minus_neg=no # Calculate 1--1
>>
>> Another case where "passing" the test means accepting MySQL's version of
>> reality instead of SQL92's. All the SQL-compliant DBMSs think -- is a
>> comment introducer

> So does MySQL -- when the '--' is followed by a space.

Considering how much we got ragged on for not being perfectly compliant
with SQL-spec handling of comments (up till 7.0 our parser didn't
recognize "--" as a comment if it was embedded in a multicharacter
operator --- but we knew that was a bug), I don't have a lot of sympathy
for MySQL unilaterally redefining the spec here. And I have even less
for them devising a test that can only be "passed" by non-spec-compliant
parsers, and then deliberately mislabeling it to give the impression
that the spec-compliant systems are seriously broken. How about
labeling the results "Fails to recognize -- comment introducer unless
surrounded by whitespace"?

Anyway, I am pleased to see you trying to clean up the mess.
Good luck!

regards, tom lane