Yesterday, during some excellent conversations with the XtC, I found myself thinking about the “one assertion per test” rule which Dave Astels proposed, and which many people advocate for Test-Driven Design.
I think it’s important that each method in a test should test only one thing, but sometimes I find it’s more helpful to use a couple of asserts to do that. For instance:
public void testShouldHavePopulationOfOneWhenOnePigeonAdded() {
PigeonCoop coop = new PigeonCoop();
coop.add(new Pigeon());
assertEquals(1, coop.getPopulation());
}
public void testShouldHavePopulationOfTwoWhenTwoPigeonsAdded() {
PigeonCoop coop = new PigeonCoop();
coop.add(new Pigeon());
coop.add(new Pigeon());
assertEquals(2, coop.getPopulation());
}
could be replaced by
public void testShouldHavePopulationEqualToTheNumberOfPigeonsAdded() {
PigeonCoop coop = new PigeonCoop();
coop.add(new Pigeon());
assertEquals(1, coop.getPopulation());
coop.add(new Pigeon());
assertEquals(2, coop.getPopulation());
}
The names of the methods at the top describe ways in which to verify that the PigeonCoop, once coded, works as expected. The name of the method at the bottom describes an aspect of the behaviour of a PigeonCoop (and serves as a reminder to myself that ‘should’, while magic, doesn’t automatically lead to Behaviour-Driven Design.)
Liz Keogh, lunivore 
The purpose of TDD/BDD is not to cover the maximum possible in one test. It is to specify behavior (what you need) in very small incremental steps. So, first you think how to verify that it returns the right thing for zero elements. This means you create a test for empty collection. Then, you wan to implement when we have one element – you create the test for one element. The third test would be for more than one. You implement then the solutions for more than one.
Of course, if you already have the implementation and you are asking yourself why you need all this tests when you can verify everything in one test, then probably it is not so obvious why “one assert per test” is a good thing.
The other very important thing about TDD/BDD is fast feedbacks. So, if you have one test and it fails, you know that the method that you test does not work any more. However, if you have three tests for this one method, and if only one of them fail (when is empty/or null for example) then you immediately know what exactly is the problem.
When the examples are so simple, we can loose the point of development with very small incremental steps and fast feedbacks. Anyway, you can take a look at BDD article by Dave Astels, if you have not done this already.
http://blog.daveastels.com/?p=53
I grow less convinced about BDD by the day. Why? It seems to be fostering a move away from the notion of the class or feature under test towards an emphasis on, as the previous anonymous poster put it, “the method that you test.” Now I like the idea of TDD as helping us come up with better designs. But I find this new focus on ‘magic’ rules such as: use “should” or only 1 assert per test (even when disguised under the banner of generativity) to be crippling what started out as a very nice idea.
When Dan first explained this to me at the Old Bank of England the emphasis was on the creation of multiple nested feedback loops from the customer tests all the way down to the unit tests. The idea of specifying things (often in a coarse grained manner) that had meaning to the customer and then drilling down to ever more fine-grained specifications seems to have been lost. The idea of only having 1 assert per test also violates one of the more useful pieces of advice in Kent Beck’s book on TDD which is that you vary the size of the steps you’re taken based on your current confidence level. So if you’re writing something very simple you might start out with relatively coarse unit tests but if you run into difficulty you can start writing more fine-grained tests.
I also wonder what happens when you run into a bug. Do you write 1 test that repeatedly exposes the bug and perhaps has to have multiple asserts in it? Or do you write lots of little tests which taken together specify the missing behaviour that would prevent the bug from happening? If you choose the latter then how do developers who later work on the codebase work out that this particular bunch of tests deal with this particular bug which only happens in this particular scenario?
We won’t even go into how discredited the strong Sapir-Whorf hypothesis is amongst linguists…
–ade
I’ve read the article by Dave Astels, and have spoken extensively with Dan North on BDD.
I developed the above code whilst thinking, “The behaviour should be present where there’s one pigeon. Now let’s implement that with the smallest thing possible – if there are pigeons, return one. Now let’s describe the behaviour more fully; it should also work with two pigeons.”
I try never to have an implementation before I write the test / design.
It is not about “the method you test”, it is about “the behaviour that is required”. The size of the steps has nothing to do with BDD. 1,2…n verifications/asserts, whatever it takes to verify the behaviour does what it should do.
You don’t have to use the word should to be doing BDD, however it gives you a consistent way of specifying what a System, Service, Component, Class etc Should do.
do you write lots of little tests which taken together specify the missing behaviour that would prevent the bug from happening
I write one method with a few asserts that do that. Or, if the bug has come about as a combination of behaviours, I work out which of the many system components should take charge of those modes of behaviour and act as a referee, then I define the manner in which the components respond to the referee.
If you choose the latter then how do developers who later work on the codebase work out that this particular bunch of tests deal with this particular bug which only happens in this particular scenario?
Well, there’ll be two sorts of automatic regression testing; one which describes the behaviour of the code, and one which describes a scenario within a story. So the scenario can be modified to describe exactly that particular bug (extra outcomes of “this should happen” or “this shouldn’t”).
I would expect that if the code in which the bug was found can’t obviously behave in a way which makes sense then it may be that something else takes responsibility for it.