Re: meson and check-tests

On Fri, Sep 20, 2024 at 6:25 PM Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com> wrote:
>
> Hi,
>
> I’ve been working on a patch and wanted to share my approach, which
> might be helpful for others. The patch removes the '--schedule' and
> '${schedule_file}' options from the regress/regress test command when
> the TESTS environment variable is set. Instead, it appends the TESTS
> variable to the end of the command.
>
> Please note that setup suite tests (at least tmp_install and
> initdb_cache) should be executed before running these tests. One
> drawback is that while the Meson logs will still show the test command
> with the '--schedule' and '${schedule_file}' options, the actual
> command used will be changed.
>
> Some examples after the patched build:
>
> $ meson test --suite regress -> fails
> $ TESTS="create_table copy jsonb" meson test --suite regress -> fails
> ### run required setup suite tests
> $ meson test tmp_install
> $ meson test initdb_cache
> ###
> $ meson test --suite regress -> passes (12s)
> $ TESTS="copy" meson test --suite regress -> passes (0.35s)
> $ TESTS="copy jsonb" meson test --suite regress -> passes (0.52s)
> $ TESTS='select_into' meson test --suite regress -> fails
> $ TESTS='test_setup select_into' meson test --suite regress -> passes (0.52s)
> $ TESTS='rangetypes multirangetypes' meson test --suite regress -> fails
> $ TESTS='test_setup multirangetypes rangetypes' meson test --suite
> regres -> fails
> $ TESTS='test_setup rangetypes multirangetypes' meson test --suite
> regress -> passes (0.91s)
>
> Any feedback would be appreciated.
>

hi. Thanks for your work!
I do find some issues.

TESTS="copy jsonb jsonb" meson test --suite regress
one will fail. not sure this is expected?

in [1] you mentioned "setup", but that "setup" is more or less like
"meson test --suite setup --suite regress"
but originally, I thought was about "src/test/regress/sql/test_setup.sql".
for example, now you cannot run src/test/regress/sql/stats_ext.sql
without first running test_setup.sql, because some functions (like fipshash)
live in test_setup.sql.

so
TESTS="copy jsonb stats_ext" meson test --suite regress
will fail.

to make it work we need change it to
TESTS="test_setup copy jsonb stats_ext" meson test --suite regress

Many tests depend on test_setup.sql, maybe we can implicitly prepend it.
Another dependency issue. alter_table depending on create_index.

TESTS="test_setup alter_table" meson test --suite regress
will fail.
TESTS="test_setup create_index alter_table" meson test --suite regress
will work.

[1] https://www.postgresql.org/message-id/CAN55FZ3t%2BeDgKtsDoyi0UYwzbMkKDfqJgvsbamar9CvY_6qWPw%40mail.gmail.com