Liz Keogh, lunivore

We write scenarios to help us know when we’re done… not

I’ve been quite happily espousing the idea that we write scenarios in order to help us (devs) know when we’re done, which helps drive our design, and everything else is a lucky by-product.

Dan referred in our OOPSLA tutorial to something he calls ‘Beer Driven Development’; how soon can I get down the pub? As a business analyst, it helps to assume that all devs are horribly lazy and want beer, so if you haven’t specified the scope of the problem, you won’t get that bit of the solution you’re after. Conversely, if you do include some behaviour in the scope of a scenario, you’ll most likely get the simplest thing that could possibly work. This is good, because it cuts down the number of bugs.

There are a couple of other reasons for writing scenarios: regression, and documentation. It turns out (from feedback at OOPSLA) that a lot of people care about automated regression tests. It’s a nice way for devs to know that we’re still done, and as long as the QAs have confidence in the automated tests, they don’t have to manually cover that ground. (In practice I’ve found that they usually do, just to make sure that the tests themselves aren’t faulty, but less often than they would.)

BDD, it turns out, lends itself quite nicely to regression testing.

A scenario is a set of steps

Given <some context>, when <some event occurs> then <some outcomes should result>.

There may be more than one context, event and outcome. It may also be useful to exercise the behaviour a few times in the same scenario, so perhaps there will be more than one event-outcome set.

Scenarios are regression tests once the code that makes them run has been written

Sometimes, though, we like to change scenarios or acceptance tests for the specific purpose of regression testing. This is because the frameworks that we have to test the UI are horribly slow.

I’ve worked on one project where, while running the suite of 500 Fitnesse scenarios, the CPU buzzed at 100% and the whole suite took 5 minutes to run. A lot of technical jiggery-pokery was needed to get this to happen, but if you can do it, you don’t need to differentiate between scenarios and regression tests (the one becomes the other as soon as the code that makes it run is in place). There are some frameworks which are faster than others – Simon Stewart’s WebDriver, and the tiny Swing harness packaged with JBehave, which I’m still incredibly proud of – but generally these fast frameworks aren’t mature, or full-featured yet. And, even if they are, once you start getting above 500 scenarios you may still want to think about cutting down the length of time it takes to get feedback from your suite.

Acceptance tests are hard to read, which makes it hard to merge them

I’ve seen (and written) a number of acceptance tests which look something like this (only they weren’t using pretty fluent interfaces, so they were even worse):

I can refund a customer’s purchase

TillScreen.addItem("Panatachi Television 3X").pay("2000").with(new CreditCard("2345123478902468")).done();
int originalNoOfTelevisionsInStock = StockScreen.count("Panatachi Television 3X");
Money originalBalance = FinanceScreen.getBalance();

assertThat(StockScreen.count("Panatachi Television 3X"), eq(originalNoOfTelevisionsInStock));
assertThat(FinanceScreen.getBalance(), eq(originalBalance - 2000));

I can replace a customer’s purchase

TillScreen.addItem("Panatachi Television 3X").pay("2000").with(new CreditCard("2345123478902468")).done();
int originalNoOfTelevisionsInStock = StockScreen.count("Panatachi Television 3X");
Money originalBalance = FinanceScreen.getBalance();

assertThat(StockScreen.count("Panatachi Television 3X"), eq(originalNoOfTelevisionsInStock - 1));
assertThat(FinanceScreen.getBalance(), eq(originalBalance));

Did you actually read those, or did you just skip to this line? Chances are that you just glossed over them. I would. BDD helps clear this up when the scenarios are written, so instead of this we’d have:

I can refund a customer’s purchase

Given that a customer purchased a Panatachi Television 3X
and a Panatachi Television 3X costs $2000and he paid with credit card number 2345123478902468
When I search for the most recent bill with credit card number 2345123478902468
and I refund the Panatachi Television 3X
Then the stock of Panatachi Television 3X should be unchanged
and we should have $2000 less.

I can replace a customer’s purchase

Given that a customer purchased a Panatachi Television 3X
and a Panatachi Television 3X costs $2000
and he paid with credit card number 2345123478902468
When I search for the most recent bill with credit card number 2345123478902468
and I replace the Panatachi Television 3X
Then the stock of Panatachi Television 3X should be one less
and we should have the same balance.

(If you think it’s impossible to write code this way, take a look at the latest RSpec features, brought to you by David Chelimsky. I’m completely blown away by this, and can’t wait to convert all my breakable toy scenarios to RSpec running on JRuby. Anyway…)

It’s easier to merge scenarios

With BDD, it’s fairly easy to see which contexts, events and outcomes are common between the two scenarios, and we can imagine a situation in which we could exercise both features associated with these stories.

You can see that the context of the scenarios both start with a customer owning a previously-purchased Panatachi television. In most frameworks, this is accomplished by running the scenario in which a customer purchases a television; we’re using a scenario as the context for another scenario. Running scenarios takes time! Wouldn’t it be great if we could just run this once?

To make these into one scenario, we need the outcome from one to resemble the context of the other, then we don’t need to set up that context again. In the same way that we’d refactor code to look similar, we can refactor the context of the first to be identical to the outcome of the second.

Interestingly, as I come to do this, I realise we’re missing a couple of contexts! They’re implicit in both scenarios – what’s the actual balance? and the actual stock levels? I’ve also swapped the scenarios around, because that’s the order they’ll happen in now:

I can replace a customer’s purchase

Given that a customer purchased a Panatachi Television 3X
and our balance is $23150
and we have 10 Panatachi Television 3Xs in stock
and he paid with credit card number 2345123478902468
When I search for the most recent bill with credit card number 2345123478902468
and I replace the Panatachi Television 3X
Then we should have 9 Panatachi Television 3Xs in stock
and our balance should be $23150.

I can refund a customer’s purchase

Given that a customer purchased a Panatachi Television 3X
and our balance is $23150
and we have 9 Panatachi Television 3Xs in stock
and a Panatachi Television 3X costs $2000
and he paid with credit card number 2345123478902468
When I search for the most recent bill with credit card number 2345123478902468
and I refund the Panatachi Television 3X
Then we should have 9 Panatachi Television 3Xs in stock
and our balance should be $21150

So, now we can merge the two scenarios:

I can replace or refund a customer’s purchase

Given that a customer purchased a Panatachi Television 3X
and our balance is $23150
and we have 10 Panatachi Television 3Xs in stock
and he paid with credit card number 2345123478902468
When I search for the most recent bill with credit card number 2345123478902468
and I replace the Panatachi Television 3X
Then we should have 9 Panatachi Television 3Xs in stock
and our balance should be $23150.
When I search for the most recent bill with credit card number 2345123478902468
and I refund the Panatachi Television 3X
Then we should have 9 Panatachi Television 3Xs in stock
and our balance should be $21150

Trying to do this with code, instead of English, sucks. It’s hard to see the difference between the setup of a context and the occurrence of an event. It’s a lot easier if the code is written in English, to the extend that most people will skip that second step and move straight to the third.

Even so, we’ve lost some of the emphasis. We don’t want to know that the balance is $23150. We want to know that the balance is the same. The implicitness of the two missing contexts was actually quite beautiful. Wouldn’t it be lovely if we could bring those back?

I want some new features!

The scenario I really want looks like this:

I can replace or refund a customer’s purchase

Given that a customer purchased a Panatachi Television 3X
and he paid with credit card number 2345123478902468
When I search for the most recent bill with that credit card number
and I replace the Panatachi Television 3X
Then we should have 1 less Panatachi Television 3X in stock
and our balance should be the same
When I search for the most recent bill with that credit card number
and I refund the Panatachi Television 3X
Then we should have 1 less Panatachi Television in stock
and our balance should be $2000 less.

So, as a scenario writer (aka BA for happy paths, or QA for full regression, or some combination with a dev to help turn it into code):

• I want outcomes to be able to record the state of the world before the events that will result in those outcomes, so that I can verify comparative outcomes and retain meaningful emphasis (This is possible in JBehave, but horribly ugly and ties the Givens to the Outcomes – I don’t want to do that!)
• I want all scenario steps to be aware of the world in which they are running, so that they can share aspects of the world instead of duplicating them (‘that credit card number’ instead of ‘2345123478902468’. JBehave already does this, but I’m not sure if RSpec does. Will ask…)
• I want to be able to associate a scenario with more than one story, so that I can change the scenarios for several stories into a single regression test (RSpec lets you tag things; JBehave doesn’t. Yet.)

As a regression tester (aka dev) being asked to save QA some work:

• I want to be able to run scenarios as well as stories (JBehave doesn’t do this yet).
• I want to be able to associate a scenario with several features, benefits, etc., so that I can test a feature or check that a benefit is still being provided (see tagging, above.)
• I want to have the option to check for the outcome of a scenario being used as the context of another scenario, so that I know whether I need to run them or not. (Since a scenario might be used as the context for a scenario that’s being used as a context, etc., at some point we are likely to find ourselves in a world we can use. Dan says this is a ‘backwards-chaining rules engine’ and is quite excited by the idea. He also says this is incredibly dangerous and will only work for Given/When/Then, not Given/When/Then/When/Then, so it should be used with some thought.)

Fortunately, as well as being a scenario writer (at least for my breakable toys) and a regression tester, I’m a JBehave dev, so I have some control over when I get these features. Looks like I have some work to do.

Someone on the XP thread suggested, “BDD is just TDD done well with different words.”

Here’s my take.

An application is the end-product, and the value of the application is delivered through its behaviour, as perceived through its user interface.

BDD uses descriptions of behaviour and executable examples (or exemplars; examples carefully chosen to illustrate a particular behaviour). These exemplars, whether in code or plain English, are an aid to communication, to driving a clean, simple design, to understanding the domain, to focusing on and delivering the business value, and to giving everyone involved clarity and confidence that the value is being delivered.

Is doing the above merely ‘doing TDD well’? Maybe it is, but I think that the words do help to change the focus; also, my experience (amongst others) is that some people who struggle to do TDD well find BDD relatively easy.

Here are some of the practices I use in BDD.

A story is a narrative from a customer in a particular role, describing a particular feature and a value that will be delivered through the use of that feature. Scenarios associated with the story will be executable, either manually or through automation, once the feature delivers the value.

(This isn’t a practice; it’s just how stories and scenarios work.)

Making the scenarios and examples executable allows us to know when the value has been delivered, at each level of code. Maintaining those examples allows us to continue to have that certainty as other features are developed.

(This isn’t a practice; it’s just how good tests work. But without using the word ‘test’.)

Each level of code in the application delivers value to some other piece of code, through its own behaviour and its own interface (which could just be the externally visible methods, or an actual java-style Interface). Eventually the sum of those values is delivered to the user interface, which delivers the value to the customer.

(This isn’t a practice; it’s just how software works.)

Describing these stories and behaviours in English allows more people to understand the code and talk about it than otherwise might. For developers, it encourages the use of English in place of Geek. This aids communication.

Driving behaviour from the user interface downwards, and focusing on each unit of code, its responsibility to deliver its value to another, and its ability to use other code to deliver value to itself, minimises non-valuable code and allows developers to split responsibilities easily. This results in a clean, simple design.

Capturing common contexts, events and outcomes, and using these as part of the (coded) domain language, helps us to understand the domain.

Coding the story and scenarios (exemplars at the user interface level, automated at the closest possible level to that interface) helps us focus on, and deliver, the desired value.

Using English for executable scenarios and examples means that everyone, including the customer, can understand which values are delivered, which have been lost (either deliberately or otherwise), and which are yet to come. This helps to give everyone clarity and confidence.

(These are some things I practice as part of BDD.)

I believe this is more than just ‘doing TDD well’. I think it’s ‘doing TDD and a number of other practices related to professional software development well’. The five practices above might conceivably be done in addition to TDD, but they’re not part of any definition of TDD I’ve seen.