There’s been a bit of a twitter storm recently, prompted by Cashto’s blog, Uncle Bob’s response, and Justin’s excellent riposte.
So, here’s my response to Cashto’s post. I value unit-level BDD hugely, and it’s fairly intuitive for me. So I’m also going to try putting in some hints and tips about how to get this right, as well as some alternative opinions that I’ve come across recently which surprised me.
I’ll be mixing language here, so that I can respond to Cashto’s language without confusing matters. If you want to use the language of BDD, see my last post.
I’ve snipped bits of your blog, Cashto, because this is already going to be a very long post. Hope that’s OK. I’ve kept your sections in to make it easier for people to read.
(You can skip to the bit where I say it’s Okay not to write unit tests if you want.)
But Unit Tests Work For Me!
First, are you sure you’re really unit testing? Unit testing is all about testing “units”—independent pieces of logic. … If they are any dependencies, they are mocked away. Do your tests do that?
Mostly, yes. I don’t tend to mock out domain objects – I use a builder pattern instead. I’ve also written integration tests which talk to the database or file system.
That’s what I thought. They’re not unit tests, they’re just tests.
I think of them as descriptions of behaviour, together with some examples which illustrate that behaviour. I’m less worried about whether they’re testing anything than I am about whether people can change the code safely and work out what went wrong, if and when it does go wrong.
You probably work on a project that lends itself to easy testing, and your platform likely has great tools for the job.
If you can reuse units of code and and throw an exception when something is false, you’ve got a tool for the job. All the rest is just reporting, and making feedback faster.
You also have great intuitions where the right level of testing is—or if not, you’re at least at that stage in your career where you make lots of little mistakes and the tests you write actually help you catch them.
When I learnt TDD, I was at that stage in my career where I didn’t quite get OO and good design. I was lucky enough to come across BDD fairly early in my TDD learning, and that really helped me realise that TDD isn’t about testing. Most of the confusion I can see in your post comes from that word, “test”. I value BDD because it makes me think about the responsibility of my classes, and where I can appropriately delegate other responsibility.
College students, for some reason, seem especially enthusiastic about unit testing.
Unlearning is much harder than learning. Once a senior developer whom I respect hugely stormed out of the room because I found it so difficult to code without writing tests first, and it was slowing him down. Eventually we realised that I was using them to understand the context of what we were doing. After that I was able to contribute much more productively. If someone really likes their unit tests, they may be doing the same thing – just trying to understand your thoughts and write them down.
But once somewhere around level four of the Dreyfus model of skill acquisition, unit tests start to lose their effectiveness. In fact, they might even be what’s holding you back from further growth.
Training wheels are a best practice—for learning how to ride a bicycle. They’re not a best practice for riding the tour de France.
As I’ve become better at BDD, I’ve started becoming more pragmatic about where I write tests and where I don’t. On my toy projects, if a class can be tested by cursory inspection and it’s covered by something somewhere else – an acceptance test or scenario, maybe – then I might skip it. If in doubt, I tend to write the test. I’ll certainly write something if there’s someone else on the team who needs to understand why I wrote the code I’m about to write. I also use them to help me with the design.
Unit Tests Give Me Confidence When Refactoring!
Unit tests tend to overspecify behavior—they test implementation details that don’t matter, rather than fixed contracts that do.
And they’re too fine-grained.
Many of them will not even survive the <refactoring> process at all. That’s what refactoring means. I’ve never known a significant refactoring which didn’t require the tests to be majorly reworked or even rewritten.
Sometimes I find this is true. It’s not so painful, though, because the design that I’ve created through BDD is simple enough that my class is no bigger than my head, and has maybe four aspects of behaviour. It’s fairly easy to change that behaviour, and the full sentences that I use to describe what I’m testing makes it easy to see what old behaviour I should be changing.
A test that needs to be updated every time the product changes is not really a test at all.
I agree that any such test isn’t really useful. The only reason for having tests is to help you change the product – otherwise you might as well manually test it, treat it as fragile and never change it again. If tests aren’t helping make change easy, there’s something wrong.
Unit Tests Catch Bugs!
Sure, on occasion you remember to test “the failure case”—the caller passed in null or a negative integer as an argument. Never mind that null or a negative argument is an assertable precondition that could never happen in production anyways.
Oh, yes. Don’t do that, people. The rest of the team are bright, intelligent people; if you give them a chance to understand the code you just wrote, they’ll use it correctly. Writing tests to describe why your code is valuable and how to use it can really help here.
It’s also not uncommon for the tests to have the same bugs as the product. And why not? The same person wrote both.
Try pair-programming, and getting one person to write the tests which the other person codes. I’ve certainly blinded myself to my own bugs on my toy projects before. Fortunately my automated scenarios or my manual testing tend to catch this.
Unit testing is no substitute for adversarial testing.
Absolutely, which is why experienced, professional testers are so valuable. They think of all the scenarios we missed. They behave like users determined to break your app. I love those sneaky buggers.
There are so many kinds of failures unit testing can’t find for you … the majority of your bugs are bigger than that, aren’t they?
They do help me fix them more quickly.
Unit Tests Improve My Design!
If anything, unit testing encourages some pretty questionable practices, like making private methods public just for so they can get the code under test…
I keep seeing this. I even saw someone do this in Programming with the Stars. Don’t do this! Write an example of how you’re going to use the code. It’s not about testing individual methods; it’s about the behaviour of an object as a whole. If you need more than one method for a class to be valuable, show how you’d use them in conjunction.
or creating zillions of interfaces for mocking purposes, interfaces that leak implementation details like water leaks through a sieve.
Of course, if you’re writing tests first, you’ve got the interfaces because you have no idea what’s going to implement them, so you’re completely independent of the implementation. Imagine it’s all done by pixies.
Its heart is in the right place when it comes to its rabid stance on decoupling.
TDD as the Cujo of the Agile world…
But there’s a lot of things unit tests don’t teach. You’d never get to Tell Don’t Ask with unit tests alone —in fact a great number of tests involve introspection of state, often in encapsulation-breaking ways.
I do sometimes find I have to revisit the way I’m using my collaborators, once I’ve actually tried to use them from the real code. This is the only time I’ve found that encapsulation breaks; I’m dependent, at least a little bit, on the implementation. I’ve not found this to be much of a problem, though. More modern, BDD-style mocking frameworks such as Mockito (Java) or Moq (.NET) also help to keep things easy to understand and change.
At best, unit testing is a weathered signpost saying “good design practices are somewhere over there”. But it’s no substitute for actually knowing those practices.
BDD, and the language of BDD, helped me to learn those practices. I also recommend Martin Fowler’s “Refactoring” and Eric Evans’ “Domain Driven Design”.
Many are beginning to discover that functional programming teaches far better design principles than unit testing ever will. … it’s shocking the number of code construction bugs which relate somehow towards mutable state
I hear this too. I’ve been trying to learn languages like Haskell. Maybe I should put more effort in here.
If You Can’t Test It, You Can’t Ship It!
Unit testing is not the only sort of testing out there. Don’t make the assumption that if a product isn’t unit tested, then it’s not tested at all.
For some people, in some contexts, it’s okay not to write unit tests. I’ve spoken to a few people who haven’t yet learned to write unit tests, but who are writing excellent code, usually with automated scenarios that test the whole system. This includes one better known member of the Kanban community, who’s been delivering code quickly and successfully for years. I certainly won’t sneer at any team or individual who can deliver working software while enabling change, however it’s done.
Now I’ll give you that some tests pay for themselves. Some of them are so good they even pay a never-ending stream of dividends. But then again, tests can have negative ROI. Not only do they cost a lot to write, they’re fragile, they’re always broken, they get in the way of your refactoring, they’re always having you chase down bogus failures, and the only way to get anything done is to ignore them.
Rather than discouraging people from writing tests at all, I’d encourage them to look at how to get the first kind of tests instead of the second.
But It’s The Corporate Standard!
Then your organization is dysfunctional.
Sorry to be so blunt, but there it is. If it’s any consolation, a lot of organizations have the same dysfunction: they don’t trust their employees to do the thinking. They think they know better than you, and they don’t.
Or, they may recognise that people in their organisation are still learning, and are putting these standards in place to ensure that the team have the space, time and motivation to learn. They may also recognise the value of tests in documenting code, allowing them to move developers between teams more easily, bring in new people, etc.
Good organizations know there are no best practices that apply regardless of context. … It’s a short walk from “the only practice we know” to “the best practice, period”, and before you know it, you’ve blinded an entire organization to better ways of doing things.
Oh, yes. Don’t do that. Question everything, challenge everything – but try it out first.
So What Are You Saying, Man?
I am suggesting that you take a good hard look at the time you’re spending and ask yourself what benefit you are really deriving. …There are so many other ways you can find bugs with less effort.
Possibly. Unit testing isn’t actually about testing, though. The word “test” is such a misleading term; it’s not just about finding bugs. It’s about preventing the bugs from happening in the first place.
Acceptance tests are great too.
The closer to production you get, the more important the feedback loops are. The most important feedback loop of all, of course, is putting the code into production and seeing if it’s as valuable as you think. (I’m learning about other important reasons for doing this too.)
Don’t beat yourself up trying to get 100% code coverage.
Do try to make sure that the rest of your team can find value in your code, understand how to use it and change it safely.
Hope this helps,
It did. These are problems with TDD that I come across frequently. Pretending they’re not there isn’t as useful as addressing them, so I hope my response has helped too. Thanks, Cashto.
Liz Keogh, lunivore 