Re: BUG - broken "make check" if different options

>> As I was investing playing around with blocksize, I noticed that some test
>> cases under "make check" vary depending on compilation parameters, as
>> they:
>
> There has never been any expectation that the regression tests would
> pass exactly no matter what the environment. If we tried to make them
> do so, we'd end up restricting the scope of testing so much as to be
> nearly useless.

Hmmm... ok, so this is a feature.

I would have thought that tests about functional results should always
pass, no matter what the environment, provided the environment allows to
run the test, so basicaly queries should return deterministic results,
implying an ORDER BY when there is more than one row in the output. So for
me this part looks more like a bug than a feature.

For tests about plans, this is less obvious. Maybe test settings should
control the environment enough so as to warrant deterministic results
(say, tell the planner to assume this and this and this, and what is your
plan ?). That would also help to test plan decisions with more extreme
hardware. However, this would probably imply a test infrastructure beyond
what is currently available. So I would be more ok for that part as a
feature.

Also, this means that changing the default block size is basically never
tested, otherwise the buildfarm would be reder than it is.

--
Fabien.