Monday, August 23, 2010

The Marginal Unit of Testing

Dear Blogspot,

Here's a puzzle for you. If society benefits from the availability of water, why is a cup of water not provided everywhere? Sure, they will offer you one at eating establishments, but why not at bookstores, or electronics stores? Granted water is not free, but if it is so necessary for human survival, you'd think we would demand a glass upfront at every venue, just like we demand air to breathe wherever we go.

The reason we don't has to do with the costs versus the marginal benefit. Radio Shack would need to spend capital and recurring expenses to be able to offer something that most people would not even want to consume were it offered there. Meanwhile, the demand for drinks at restaurants is extremely high, due to the complementary good of Food that is consumed there. Put simply: the marginal benefit of that next glass of water, at most venues, is well below the cost of provision. Not at every venue, but at most.

This important economic insight applies equally to an important (and recently Hallowed and Revered, thanks to T.D.D.) object such as the Unit Test. A unit test, of course, is programming code written to test the functionality of other programming code. It is distinguished from integration or functional testing precisely because of the small segments of code that are the target of the unit tests. Unit Tests are almost always written by the same programmer who wrote (or will write) the code being tested.

Like water, unit tests are important. Also like water, the provision of unit tests is not free. However, unlike water, far too many developers (especially the last 5-10 years) would be Horrified to discover even one "venue" where unit tests are not provided for. This reflects, in my humble opinion dear Blogspot, a failure to think on the margin.

Like with Water and Radio Shack, spotting the most cost-inefficient places for unit tests is pretty easy: getters and setters, code that does trivial calculations, code that generates logs for developer purposes, or code whose failure is more cheaply spotted by integration testing (such as user interfaces), etc. Below this it gets rather fuzzy -- one might argue that code that has a high call count, has numerous dependencies, is modified frequently, has failed previously, or that performs a function whose failure would be disastrous are all great candidates for unit tests.

Either way, treating Unit Tests like Water and reasoning that "because they are important, they must be equally demanded in all situations" is both fallacious (see the Fallacy of Composition & Division) and a wasteful use of resources.


1 comment:

  1. I would have to agree that writing them in places where they are not usefull is a waste. Getters and Setters (unless there is other code in there that serves a usefull purpose) are not good places to write test code around.

    Logging, hmmm.... now there may be a difference of opinion. Recently a developer changed some logging methods where I work and half the messages (the ones that were most valuable) stopped being written to the log. Ok, I think it might have been good to test that one before pushing it out. Perhaps a once written test that gets run on checkin would be valuable here.

    however, one example does not contradict what you are saying. As with most things in programming, following a rule for the sake of following a rule is a waste.

    ReplyDelete