Re: The real reason why TAP testing isn't ready for prime time

From: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Peter Eisentraut <peter_e(at)gmx(dot)net>, Andrew Dunstan <andrew(at)dunslane(dot)net>, PostgreSQL mailing lists <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: The real reason why TAP testing isn't ready for prime time
Date: 2015-06-18 06:52:29
Message-ID: CAB7nPqTEqriEXH7uM0LiMcXnXSHLD5Kuzm5PwLtHjBgtjNKEbg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Jun 15, 2015 at 8:26 AM, Michael Paquier wrote:
> hamster is legendary slow and has a slow disc, hence it improves
> chances of catching race conditions, and it is the only slow buildfarm
> machine enabling the TAP tests (by comparison dangomushi has never
> failed with the TAP tests) hence I would prefer thinking that the
> problem is not specific to ArchLinux ARM. In this case the failure
> seems to be related to the timing test servers stop and start even if
> -w switch is used with pg_ctl, particularly that PGPORT is set to the
> same value for all servers... Still, for the time being I don't mind
> disabling them and just did so now. I will try to investigate further
> on the machine itself.

So, I have been doing some progress on this thing.. And wrote the
patch attached that improves the way TAP tests log their output using
IPC::Run::run. This simply creates a log file as regress_log/test_name
for each test run, capturing all the command outputs. For example on
current HEAD, some commands of pg_ctl are simply silenced or have
their output redirected to /dev/null. With this patch, all their
output is redirected to the log file. This is as well compatible with
Windows, and logging of pg_rewind tests is unified with the new
structure of TestLib.pm. I am going to apply that to hamster and see
what is causing the error reported.

I think that it would be useful as well to improve the buildfarm
output. Thoughts?
Regards
--
Michael

Attachment Content-Type Size
20160618_tap_improve_logs.patch application/x-patch 17.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Cédric Villemain 2015-06-18 10:05:32 Re: Auto-vacuum is not running in 9.1.12
Previous Message Peter Eisentraut 2015-06-18 03:45:02 Re: 9.5 make world failing due to sgml tools missing