Unit Testing (was: Bugs vs. unit tests)

[thelusiv]

I recently read a programming book that has a good piece of advice in it: "Find Bugs Once". The idea is, whenever a bug is reported, write a unit test that correctly detects the bug and fails. Then if a regression occurs and the bug resurfaces, it will be caught immediately by the test rig.

Our testing setup should be pretty easy to automate. I'm thinking about running tests nightly on the main development branch (currently trunk). I think I can make it output a web page that displays which tests failed (if any), and the details of the failure. If anyone has suggestions for the best way to implement this, please post them.

Of course, this will only work if tests actually get written. I will try to write some more tests for classes that don't have any yet. I won't be able to test every class on my own, however...so is anyone interested in helping with writing more tests?

edit: I should have mentioned this more clearly, but I wrote a new SCons builder to automatically build a test rig for VDrift. Might as well use this thread (so I modified the title) to talk about testing in general. I wrote a new file (src/unittest.cpp) for the build that has a simple main() that runs the tests. To build the test rig, do:

Code:

scons test

More information can be found here: http://wiki.vdrift.net/Testing

[joevenzon]

Ihe idea is, whenever a bug is reported, write a unit test that correctly detects the bug and fails.

Not always possible, but I agree.

[thelusiv]

Of course, but I think it's doable in many cases, especially for lower-level classes...I think a lot of times, bugs which manifest themselves in high-level ways are often really low-level problems.

[thelusiv]

from the Auto-updater thread:

The main function in unittest.cpp might cause problems for MacOS guys. They are compiling everything in the src directory.

I think the Mac build method is the problem, really. Is there no way to exclude a file from the build with Xcode?

This is a bigger question, but why not just make the Xcode build rely on SCons? There is already OS X-specific code in SConstruct and src/SConscript to handle this, although it probably needs to be updated since it has been neglected. One of the original reasons for using SCons was that it was relatively easy to use the same build system across different operating systems. Things like special build targets (e.g. "scons test") are not available to OS X developers, and the likely result is that the tests will never get run.

A hack to work around the problem would be to just put

Code:

#ifndef __APPLE__
/* int main... */
#endif

around the main function in unittest.cpp.