Thoughts on unit testing?

Hi hackers,

The current regression tests, isolation tests and TAP tests are very
good (though I admit my experience with TAP is limited), but IMHO we
are lacking support for C-level unit testing. Complicated, fiddly
things with many states, interactions, edge cases etc can be hard to
get full test coverage on from the outside. Consider
src/backend/utils/mmgr/freepage.c as a case in point.

I guess we could consider the various xUnit systems for C[1] and have
an entirely new kind of test that runs independently of PostgreSQL. I
guess that'd be difficult politically (choosing external project,
cognitive load) and technically (global state problems as soon as you
get away from completely stand-alone components).

One idea that keeps coming back to me is that we could probably extend
our existing regression tests to cover C tests with automatic
discovery/minimal boilerplate. Imagine if you just had to create a
file bitmapset.test.c that sits beside bitmapset.c (and perhaps add it
to TEST_OBJS), and in it write tests using a tiny set of macros a bit
like Google Test's[2]. It could get automagically sucked into a test
driver shlib module, perhaps one per source directory/subsystem, that
is somehow discovered, loaded and run inside PostgreSQL as part of the
regression suite, or perhaps it's just explicitly listed in the
regression schedule with a .sql file that loads the module and runs an
entry point function.

One problem is that if this was happening inside an FMGR function it'd
be always in a transaction, which has implications. There are
probably better ways to do it.

Thoughts, better ideas?

[1] https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C
[2] https://github.com/google/googletest/blob/master/googletest/samples/sample1_unittest.cc

--
Thomas Munro
http://www.enterprisedb.com