Re: fedora core 2 postgresql regression tests fail

On Thursday 30 September 2004 00:07, John McBride wrote:
> As user postgres I try and run the tests:

> $ pwd
> /usr/lib/pgsql/test/regress
> $ time ./pg_regress.sh --schedule=parallel_schedule

> ====================================================
> 6 of 93 tests failed, 1 of these failures ignored.
> ====================================================

My aplogies for the delay, I have been very busy and have not up until a few
minutes ago been able to attempt to reproduce your problems.

I can't reproduce some of your problems. When running the tests in the RPM
way (see README.rpm-dist in /usr/share/docs/postgresql-7.4.2) (note that you
do NOT need to separately creatlang plpgsql, since the regression script does
that for you):
bash-2.05b$ ./pg_regress --schedule=parallel_schedule
(using postmaster on Unix socket, default port)
============== dropping database "regression" ==============
dropdb: database removal failed: ERROR: database "regression" does not exist
============== creating database "regression" ==============
CREATE DATABASE
ALTER DATABASE
============== dropping regression test user accounts ==============
============== installing PL/pgSQL ==============
============== running regression test queries ==============
parallel group (13 tests): boolean int4 float8 int2 char int8 text name
varchar oid bit float4 numeric
[snip]
parallel group (20 tests): point lseg time date timetz timestamp reltime
intervaltimestamptz tinterval inet path box abstime circle polygon comments
type_sanity oidjoins opr_sanity
[snip]
test create_function_1 ... FAILED
[snip]
parallel group (17 tests): select_into select_distinct select_implicit
select_having select_distinct_on case aggregates portals union btree_index
random hash_index update transactions join arrays subselect
[snip]
random ... failed (ignored)
[snip]
====================================================
2 of 93 tests failed, 1 of these failures ignored.
====================================================

=======regression.diffs============
*** ./expected/create_function_1.out Wed Mar 10 16:36:51 2004
--- ./results/create_function_1.out Mon Oct 4 13:44:19 2004
***************
*** 70,76 ****
ERROR: could not access file "nosuchfile": No such file or directory
CREATE FUNCTION test1 (int) RETURNS int LANGUAGE c
AS '/usr/lib/pgsql/test/regress/regress.so', 'nosuchsymbol';
! ERROR: Can't find function nosuchsymbol in
file /usr/lib/pgsql/test/regress/regr
ess.so
CREATE FUNCTION test1 (int) RETURNS int LANGUAGE internal
AS 'nosuch';
ERROR: there is no built-in function named "nosuch"
--- 70,76 ----
ERROR: could not access file "nosuchfile": No such file or directory
CREATE FUNCTION test1 (int) RETURNS int LANGUAGE c
AS '/usr/lib/pgsql/test/regress/regress.so', 'nosuchsymbol';
! ERROR: could not find function "nosuchsymbol" in file
"/usr/lib/pgsql/test/regre
ss/regress.so"
CREATE FUNCTION test1 (int) RETURNS int LANGUAGE internal
AS 'nosuch';
ERROR: there is no built-in function named "nosuch"

======================================================================

*** ./expected/random.out Thu Feb 13 00:24:04 2003
--- ./results/random.out Mon Oct 4 13:44:45 2004
***************
*** 31,35 ****
WHERE random NOT BETWEEN 80 AND 120;
random
--------
! (0 rows)

--- 31,36 ----
WHERE random NOT BETWEEN 80 AND 120;
random
--------
! 73
! (1 row)

===============
Now, if you'll look very carefully the create_funtion_1 failure is due to the
difference between the expected output:
Can't find function nosuchsymbol
and the generated output:
could not find function "nosuchsymbol"

This is not a problem, since it's just a different wording. The test is
checking to make sure that it can reliably fail an attempt to create a
nonexistent funtion. Minor differences in error message wording often causes
the regression test suite to trip up on trivial things, like this one.

The random test fails randomly anyway; random can present a result not between
80 and 120 on occasion. It is _SUPPOSED_ to fail in a random manner.

My box is Fedora Core 2, fully updated, with a few third party RPM's loaded,
but none of which should change the PostgreSQL results.

Now, since I am the one responsible for the regression tests being present in
the RPMset, let me make note that the instructions on the from-source
RECOMMENDED manner of running the regression tests DO NOT WORK in the RPM
installation, and never have. Instead of invoking pg_regress.sh, invoke
pg_regress directly, as I did above. The initial variable substitutions that
the from-source make does I already have done in the building process.

So, in my case, the regression tests PASS, since the two failures aren't
really failures. This is better than previous versions, where failures were
expected due to libc difficulties, locale problems, etc. Things are much
better now.

The regression tests have been bundled in the RPMset since 6.5.2 days (five
years ago). They have never been terribly reliable in the RPM installation,
but my primary intention in packaging them (which has been contested before)
was so that they were available without the source. They can (and do) run
without a source tree installed, but it isn't an easy thing, and you do have
to investigate the failures very carefully. It is just recently that Red
Hat/Fedora Project has pulled through the -test subpackage into their
distribution; previously it was not built by them (disabled in the spec file)
due to space constraints and the potential for confusion.
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute
1 PARI Drive
Rosman, NC 28772
(828)862-5554
www.pari.edu