NUnit Practice: Comment boolean asserts

(Disclaimer: I am not much of a baseball fan (as in nope, not even a little bit) -- and I am not much for jumping on bandwagons, but I needed an example, and everyone in Chicago is talking about the White Soxs these days so...)

When using asserts that make boolean checks (e.g. IsTrue, IsFalse, IsNotNull, etc), it's generally a good idea to add a comment to the assert that makes it easier to understand why its failing.

By default, boolean NUnit asserts only tell you the line on which the assert failed; specifically there isn't any context in the test output beyond the line number to indicate what failed or why. By adding an 'on failure' comment, its much easier to understand the situation just by looking at the tool output. For example, when asserting what happened last night, without a comment the assert looks like this:


If this assert had failed, it would've resulted in the following output:

TestCase 'Chicago.WorldSeriesTests.SoxWillWinGameOneByPuttingInReliefPitcher' failed:
        at Tests.SeriesTests.SoxWillWinGameOneByPuttingInReliefPitcher()

But, by adding text to explain the failure case of the assert:

Assert.IsTrue(sox.WinGameOne, "Sox should have won, but reliever didn't finish the job.");

The output would change to:

TestCase 'Chicago.WorldSeriesTests.SoxWillWinGameOneByPuttingInReliefPitcher' failed: 
        Sox should have won, but reliever didn't finish the job.
        at Tests.SeriesTests.SoxWillWinGameOneByPuttingInReliefPitcher()

This type of commenting is possible (and sometimes useful) with comparision (e.g. AreEqual) assert statements too, however NUnit provides decent output about expected versus actual values, so I generally only add failure text to them when it isn't obvious from the default nunit output.

Like all comments, this is deoderant for a smell. So, while I don't hesitate to use boolean checks, I do look for ways to change my tests to make such comments superflous by using comparison checks too (and by that I don't mean simply asserting that something equals true).

Related to this, I've noticed that doing interaction-based testing (as opposed to state-based testing) increases the number of boolean checks that I make -- i.e. if I am asserting that methods are called, windows are closed, values are set, etc, then I end up with more true/false testing than if I am checking the value of a method call, whether a data structure is correctly populated, etc. Again, this isn't to say one way is better than the other, just noting the correlation.

So, does this mean that you can't do comparisions and interaction-based testing? Of course not, just that you might need to supplement those boolean checks with other stuff. For example, I often find that as my tests increase in sophistication, I supplement saying "assert method called" with: "assert method called this many times", or "this should be the order of the parameter arguments the method receives over the next n calls", and so forth.

As usual (lately), no real axe to grind, just found myself talking about this the other day, and thought it would make a good quick entry for a Sunday.