Hi,
I've been doing a lot of tests under valgrind lately, and it made me
acutely aware of how long check-world takes. I realize valgrind is
inherently expensive and slow, and maybe the reasonable reply is to just
run a couple tests that are "interesting" for a patch ...
Anyway, I did a simple experiment - I ran check-world with timing info
for TAP tests, both with and without valgrind, and if I plot the results
I get the attached charts (same data, second one has log-scale axes).
The basic rule is that valgrind means a very consistent 100x slowdown. I
guess it might vary a bit depending on compile flags, but not much.
But there are two tests very clearly stand out - not by slowdown, that's
perfectly in line with the 100x figure - but by total duration. I've
labeled them on the linear-scale chart.
It's 002_pg_upgrade and 027_stream_regress. I guess the reasons for the
slowness are pretty clear - those are massive tests. pg_upgrade creates,
dumps and restores many objects (thousands?), stream_regress runs the
whole regress test suite on primary, and cross-checks what gets
replicated to standby. So it's expected to be somewhat expected.
Still, I wonder if there might be faster way to do these tests, because
these two tests alone add close to 3h of the valgrind run. Of course,
it's not just about valgrind - these tests are slow even in regular
runs, taking almost a minute each, but it's a different scale (minutes
instead of hours). Would be nice to speed it up too, though.
I don't have a great idea how to speed up these tests, unfortunately.
But one of the problems is that all the TAP tests run serially - one
after each other. Could we instead run them in parallel? The tests setup
their "private" clusters anyway, right?
regards
--
Tomas Vondra
