Using Scenarios for Experiment Design

In the complex domain, cause and effect are only correlated in retrospect, and we cannot predict outcomes. We can see them and understand them in retrospect, but the complex domain is the domain of discovery and innovation. Expect the unexpected! Every time we do something new which hasn’t been done before, or hasn’t been done within the given context, there are going to be complex aspects to it.

The only discovery-free project would be the same project, done with the same people, the same technology and the same requirements. That never happens!

Because of this, analysis doesn’t work for every aspect of a project. People who try to do analysis in the complex domain commonly experience analysis paralysis, thrashing, two-hour meetings led by “experts” who’ve never done it before either, and arguments about who’s to blame for the resulting discoveries.

Instead, the right thing to do is to probe; to design and perform experiments from which we can learn, and which will help to uncover information and develop expertise.

There are few things we can do to design experiments well, with thanks and credit for the strategies going to Cognitive Edge. (They have more material on this, available in their library if you sign up for free). Suggestions around scenarios are mine.

Amplification strategy; recovery strategy

For our experiment to work, we have to know how we’re going to amplify it. That may mean adding it to corporate processes, communicating it to a wider audience, automating it, releasing it to production, etc.. In the complex space doing the same thing over and over results in different outcomes because of the changing contexts, but once we understand cause and effect, we can start to test out that correlation in different or larger contexts, and develop expertise, moving it into the complicated domain.

We also need to have a strategy for recovery in case of failure. This doesn’t mean that we avoid failure!

I’ve seen a lot of people try to analyze their way out of every failure mode. One of my clients said, “Oh, but what if people don’t have the skills to do this experiment?” Does it matter? If the investment in the experiment is small enough (which is also part of being safe to fail) then all we need to know is that failure is safe; that we can recover from people not having skills. We don’t have to put everything in place to ensure success… and perhaps good things will happen from people being forced to gain skills, or using their existing skills to create a novel approach! This is the nature of experiments; that we don’t know the outcome, only that it has coherence, which means a reason for believing its impact, whatever it is, might be positive. More on that later.

If you can think of a scenario in which the experiment succeeds, can you think of how to make it succeed a second time, and then a third?

If you can think of a scenario in which it fails, can you think of how to make that failure safe (preferable to worrying about how to avoid the failure)? I find the evil hat very handy for thinking up failure scenarios.

Indications of failure; indications of success

In order to put into place our amplification or recovery strategies, we need to be able to tell whether an experiment is succeeding or failing. Metrics are fantastic for this. Don’t use them as tests, though, and definitely don’t use them as targets! They’re indicators; they may not behave as expected. We can understand the indicators in retrospect, but cause and effect won’t always be correlated until then.

As an example, one group I met decided to experiment to see if they could reduce their bug count by hiring a new team member and rotating an existing team member each week into a bug-fixing role. Their bug count started to go down! So they took another team member and hired another new person… but the bug count started to go up again.

It turned out that the users had spotted that bugs were being fixed, so they’d started reporting them. The bugs were always there! And the count of known bugs going up was actually a good thing.

Rather than thinking of tests, think of scenarios in which you can see the experiment succeeding or failing. Those things which allow you to see it – specific, measureable, relevant signs – will make for good indicators. These indicators will have to be monitored.

Rationale for Experiment

The experiment should be coherent.

This means that there should be a reason for believing the impact will be good, or as Dave Snowden puts it, “a sufficiency of evidence to allow us to progress“.

If you can come up with some realistic scenarios in which the experiment has a positive impact, you have coherence. The more likely the scenario is – and the more similar it is to scenarios you’ve seen in the past – then the more coherent it becomes, until the coherence is predictable and you have merely complicated problems, solvable with expertise, rather than complex ones.

To check that your scenarios are realistic, imagine yourself in the future, in that scenario. Where are you when you realise that the experiment has worked (or, if checking for safe failure, failed)? Who’s around you? What can you see? What can you hear? What colour are the walls, or if you’re outside, what else is around? Do you have a kinesthetic sense; something internal that tells  you that you’ve succeeded, like a feeling of pride or joy? This well-formed outcome will help you to verify that your scenario is realistic enough to be worth pursuing.

If you can’t come up with any scenarios in which you can imagine a positive impact, then your experiment is not coherent, and you might want to think of some other ideas.

Look out for a blog on how to do that with a shallow dive into chaos soon!

Posted in cynefin, evil hat, Uncategorized | 4 Comments

A Little Tense

Following on from my last blog post about deriving Gherkin from conversations, I wanted to share some tips on tenses. This is beginner stuff, but it turns out there are a lot of beginners out there! It also isn’t gospel, so if you’re doing something different, it’s probably OK.

Contexts have happened in the past

When I phrase a context, I often put it in the past tense:

Given Fred bought a microwave

Sometimes the past has set up something which is ongoing in the present, but it’s not an action as much as a continuation. So we’ll either use the present continuous tense (“is X-ing”) or we’ll be describing an ongoing state:

Given Bertha is reading Moby Dick

Given Fluffy is 1 1/2 months old

It doesn’t matter how the context was set up, either, so often we find that contexts use the passive voice for the events which made them occur (often “was X-ed” or “has been X-ed”, for whatever the past tense of “X” is):

Given Pat’s holiday has been approved

Given the last bar of chocolate was sold

Events happen in the present

The event is the thing which causes the outcome:

When I go to the checkout

When Bob adds the gig to his calendar

I sometimes see people phrase events in the passive voice:

When the last book is sold

but for events, I much prefer to change it so that it’s active:

 When we sell the last book

When a customer buys the last book

This helps to differentiate it from the contexts, and makes us think a bit harder about who or what triggers the outcome.

Outcomes should happen

I tend to use the word “should” with outcomes these days. As well as allowing for questioning and uncertainty, it differentiates the outcome from contexts and events, which might otherwise have the same syntax and be hard to automate in some frameworks as a result (JBehave, for instance, didn’t actually care whether you used Given, When or Then at the beginning of a step; it just told it there was a step to run).

Then the book should be listed as out of stock

Then we should be told that Fluffy is too young

I often use the passive voice here as well, since in most cases it’s the system producing the outcome, unless it’s pixies.

And that’s it!

Posted in bdd | 2 Comments

Deriving Gherkin from Real Conversations

The “Given, When, Then” format was originally developed by Dan North and Chris Matts, way back in 2004. It was originally intended as a way of describing class behaviour using something that didn’t involve testing. It was a way of having useful conversations about code. It turned into a way of having conversations about entire systems, with examples of how those systems behave. It was always meant to be readable by people who didn’t code (Chris Matts being a business analyst at the time) but it was never quite how people actually spoke.

In this post I want to look at ways of turning real conversations into Gherkin, while maintaining as much of the language as possible. If you’ve got any hints and tips of your own, please add them in the comments!

Get examples by asking for them

The best way to get examples is to ask someone, “Can you give me an example?”

Frequently, the people who are thinking of requirements are trying to think of all of them at once, so you’ll probably get something back that’s acceptance criteria rather than a concrete example.

For instance, if I ask:

“Can you give me an example of something that Word does?”

I might get back:

“Yeah, when I select text and make it italic it should be in italics.”

So I make it more specific.

“Can you give me an example of something you might want to make italic?”

“Sure. Let’s say I’ve lost my dog, and I’m making a poster; I want to make ‘Answers to Spot'” appear in italics.”

Usually when people come up with specific examples they’ll come up with something that’s a bit surprising or funny; something which gives you insight into why it might be valuable as well as what you can do with it, without going into how. Listen to what people actually say!

While you’re practicing capturing real scenarios in real conversations, start by writing down exactly what people say. You might come up with something like the above. Or perhaps they’ll say something like:

If I bought the microwave on discount then I bring it back for a refund, I should only get $90 instead of $100.

The closer your scenarios match the real language people use, the more readable and interesting they’re likely to be.

Draw out the Givens, Whens and Thens; don’t force them.

People don’t use “Given, When, Then” in conversation. I frequently find that:

  • people use “if” where Gherkin uses “given”
  • or they use “Let’s say this happened…”
  • or say “well, if I’ve already done X…”
  • people use “then” instead of “when”, when they’re talking about an event they’re performing
  • people often skip “then” altogether when they’re talking about an outcome, and occasionally skip “when”.

So you might get scenarios of the format:

  • Let’s say <context>. I do <event>, then <outcome>.
  • If <context>, then <event>, <outcome>.
  • If we start on <context page>, then <event>, <outcome>.

It’s all pixies.

I do a lot of work with teams that have already done significant analysis before they even get a sniff at code. For those teams, the solution is already designed; but it’s still useful to help them talk about their problem without reference to the solution.

I tell them, “It’s not a <context page>. It’s a pixie. You’re talking to a pixie, and your pixie is saying, ‘Okay, so what have you already said to me?'” And I make a very silly, high-pitched voice, which makes people laugh but also draws them away from the code and into genuine conversation.

“I’ve said that <context…>”

That’s better than “Given we start on <context page>”, because it doesn’t reference the fact that we’re using a web application, so you can always make a nice little app to do it instead. The UI is flexible, which is good, because people keep wanting to change it, especially when things are uncertain.
Or you could let the pixies do it for you.

Success vs. failure

Often people phrase steps exactly the same way, whether they’re successful or not. So you’ll get:

I submit <my application> and <look for this successful outcome>

I submit <my application> but miss out <mandatory field>

I often see this:

Given I have submitted my application successfully
When I enter my details successfully

I find it more readable to make the default event successful, and use try for unsuccessful events, so:

Given I’ve submitted my application

means you succeeded, while

Given I tried to submit my application
But it was audited and rejected

naturally flows; you recognise in the first step that there’s some other information that’s a bit different to normal. You can use this for events as well as contexts.


Use the word it, or he, she, etc. Nobody says:

Given John filled in his application
And the application meets the auditor’s regulatory requirements
When John submits the applicaton for approval
Then John should receive an email that his application has been approved…

Closer to the conversation would be:

Given John filled in his application
And it meets the auditor’s regulatory requirements
When he submits it for approval

You get the idea.

It’s OK to ignore bits in code.

If you want to make sure you can handle it, he, she or even an occasional John or the application without writing fifteen step definitions, you can turn them into arguments to the steps and then just ignore them. Call them actor_ignored or something similar so you know you won’t be using them.

It’s also OK to have steps which lend some understanding of why things are important, even though you’ve got no way of actually coding them. For instance:

Given I lost my dog

can just be an empty step. We don’t want to actually lose someone’s dog. Especially not to the pixies.

Posted in bdd | 1 Comment

A dev walks into a bar…

…and says to the barman, “I’m in the bar. I’m thirsty. I have £10.23 in my wallet.”

“Great,” says the barman. “What can I get you?”

The dev looks around. “When you take that glass and put it in front of that pump there,” he says, pointing at a pump, “you should be able to fill it full of beer.”

“Guess so,” the barman says. He picks up the glass and starts pouring the pint.

The dev points to a spot in front of him on the bar. “Given the glass is full of beer, when you put it there on the bar, you should ask me for £3.80,”

“Uhuh,” the barman says. He finishes pouring the pint and puts it in front of the dev.

“You should ask me for £3.80,” the dev says again. “If you don’t, I’m going to throw… um…” He looks around again.

“You know,” the barman suggests, “if you want to learn to use Cucumber you could just start by having an ordinary conversation first.”

Posted in bdd | 2 Comments

Using BDD as a Sensemaking Technique

A while back, I wrote about Cynefin, a framework for making sense of the world, and for approaching different situations and problems depending on how much certainty or uncertainty they have.

As a quick summary, Cynefin has five domains:

Simple / Obvious: Problems are easy to solve and have one best solution. You can categorise and say, “Oh, it’s one of those problems.”

Complicated: Often made up of parts which add together predictably. Requires expertise to solve. You can analyse this problem if you have the expertise.

Complex: Properties emerge as a result of the interactions of the parts. Cause and effect are only correlated in retrospect. Problems must be approached by experiment, or probe. Both outcomes and practices emerge.

Chaos: Resolves itself quickly, and not always in your favour. Often disastrous and to be avoided, but can also be entered deliberately (a shallow dive into chaos) to help generate innovation. Problems need someone to act quickly. Throwing constraints around the problem can help move it into complexity. Practices which come from this space are novel.

Disorder: We don’t know which domain dominates, so we behave according to our preferred domain (think PMs demanding predictability even when we’re doing something very new that we’ve never done before, or devs reinventing the wheel rather than getting an off-the-shelf library). Often leads to chaos when the domain resolves (I know least about this domain, but it’s way more important than I originally thought it was!)

By looking to see what kind of problems we have, we can choose an appropriate approach and avoid disorder.

However, we can also use BDD in conversation as a sensemaking technique!

BDD uses examples in conversation to illustrate behaviour. We sometimes call those examples scenarios, but really they mean the same thing. My favourite technique for eliciting examples is just to ask for them: “Can you give me an example?”

If the scenarios are imminent and dangerous, and we want to avoid them, we’re probably in chaos – and honestly, you won’t be having a “huddle” or a “scenario-writing session”; you’ll be having a hands-on emergency meeting. You’ll know if you’re in chaos. Don’t worry about talking through the scenarios any more. Get people who know how to stem the blood-flow into the room and throw some constraints around the problem like a torniquet (shut down that problematic server, or put up a maintenance page, or send an apology to your customers).

If the scenarios are causing a lot of discussion, and people are looking worried or confused, it’s probably because you’re in complexity. BDD is essentially an analysis tool, and analysis doesn’t work in complexity. You’ll see analysis paralysis, in which people try to thrash out various outcomes, and every answer generates yet more unanswered questions. As a check to see if you’re really in this space, ask, “Are we having trouble analysing this because it’s so new?” If so, see if you can think of a way to spike or prototype the ideas you’re generating, as cheaply as possible, so you can start getting feedback on them rather than deciding everything up-front. These are the 4s and 5s on my complexity estimation scale. We want to do these as early as possible, because they carry the most risk and the most value, so it’s very important not to push back on the business for clear acceptance criteria here! That will just end up pushing the riskiest requirements towards the end of the project, when we have less time to react to discoveries.

If BDD is working well for understanding the problem and gaining expertise in the business domain, you’re probably in complicated territory. Fantastic! BDD will work well for you here. People will be interested, and asking questions about scenarios generates a few more scenarios and helps create a common understanding. This is a 3 on the complexity estimation scale. Don’t forget to get feedback quickly anyway, because we’re all human and we all make mistakes. I tend to get devs to write the scenarios down, either during or after the conversations, since they can then get feedback on their understanding (or lack of it) early, before they even write any code.

If people are getting bored with discussion around scenarios, look to see if the problem is well-understood, or very similar to something that the team is familiar with. This is either a 2, which means it’s on the border of complicated and simple, and well-understood by people who work in that business domain, or a 1, which means it’s simple and obvious and easy to solve. You can always use Dan North’s “Ginger Cake” pattern here. Find a chocolate cake recipe that’s similar (“log in like Twitter”) and replace the chocolate with ginger (“but make them upload a photo instead of creating a username”). I find it enough just to name the scenarios here, without going into the actual steps. As a check, you can ask, “Is there anything different about <scenario> compared to <the other time we did it>?” That will help flush out anything which isn’t obvious.

The most important part of using BDD this way is to pay attention to people’s spoken and body language – bored, interested, worried or panicked, depending on the domain you’re in. I find BDD particularly good for locating the borders of complex/complicated and complicated/simple, and isolating which bits of problems are the most complex. And that’s how I use BDD across all the Cynefin domains, including the ones in which it doesn’t work!

(I’ll be teaching this and other BDD / Cynefin techniques as part of the BDD Kickstart open course with Cucumber founder Aslak Hellesøy in Berlin, 22 to 24 October – tickets available here!)

Posted in bdd, complexity, cynefin | 3 Comments

Goals vs. Capabilities

Every project worth doing has a vision, and someone who’s championed that vision and fought for the budget. That person is the primary stakeholder. (This person is the real product owner; anyone else is just a proxy with a title.)

In order to achieve the vision, a bunch of other stakeholders have goals which have to be met. For instance:

  • The moderator of the site wants to stop bots spamming the site.
  • The architect wants the system to be maintainable and extensible.
  • The users want to know how to use the system.

These goals are tied in with the stakeholder’s roles. They don’t change from project to project, though they might change from job to job, or when the stakeholder changes roles internally.

Within a project, we deliver the stakeholder’s goals by providing people and the system with different capabilities. The capabilities show us how we will achieve the goals within the scope of a particular project, but aren’t concrete; it still doesn’t matter how we achieve the capabilities. The word “capability” means to be able to do something really well.

  • The system will be able to tell whether users are humans or bots.
  • Developers will be able to replace any part of the system easily.
  • Users will be able to read documentation for each new screen.

Often we end up jumping straight from the goals to the features which implement the capabilities. I think it’s worth backtracking to remember what capability we’re delivering, because it helps us work out if there’s any other way to implement it.

  • We’re creating this captcha box because we need to tell if humans are users or bots. Or we could just make them sign in via their Twitter account…
  • We’re adding this adapter because we want to be able to replace the persistence layer if we need to. Or we could just use microservices and replace the whole thing…
  • We’re writing documentation in the week before we go live. Or we could just generate it from the blurb in the automated scenarios…

By chunking up from the features to the capabilities, we can give ourselves more options. Options have value!

But chunking down from goals to capabilities is also useful. The goals for a stakeholder’s role don’t tend to change, but neither do the capabilities for the project, which makes them nice-sized chunks for planning purposes.

Features, which implement the capabilities, change all the time, especially if the capabilities are new. And stories are just a slice through a feature to get quick feedback on whether we’re delivering the capability or not (or understanding it well, or not!).

Be careful with the word epic, which I’ve found tends to refer indiscriminately to goals, capabilities, features or just slices through features which are a bit too big to get feedback on (big stories). The Odyssey is an epic. What you have is something different.

Posted in capability red, complexity, stakeholders | 5 Comments

Discrete vs. Continuous Capabilities

A capability is more than just being able to do something.

The word which describes being able to do something is ability. I do sometimes use this while describing what a capability is, but there are connotations there that are missing.

When we say that someone is capable, we don’t just mean that they can do something. We mean that they can do it well, competently; that we can trust them to do it; that we can rely on them for that particular ability. The etymology comes from the Latin meaning “able to grasp or hold”. I like to think of a system with a capability as not just being able to do something, but seizing the ability, grasping it, triumphantly and with purpose.

Usually when I talk about capabilities, I’m talking about the ability of a system to enable a user or stakeholder to do something. The capability to book a trade. The capability to generate a report. These are subtly different to stakeholder’s goals. The trader doesn’t want to book a trade; he wants to earn his bonus on the trades he books. The auditor is responsible for governance and risk management; his goal is not to read the report, but to ensure that a company is behaving responsibly. The capabilities are routes to that.

We’re most familiar with discrete capabilities.

In order to deliver capabilities, we determine what features we’re going to try out, and at that point we start moving more firmly into the solution space. Capabilities are a great way of exploring problems and opportunities, without getting too far into the detail of how we’ll solve or exploit them.

Features, though, are discrete in nature. Want to book a trade? Here’s the Trade Booking feature. Auditing? Here’s the Risk Report Generation feature. For each of these features, a user starts from a given context (counterparties set up, trades for the year available), performs an event (using the feature) and looks for an outcome (the trade contract, the report).

For anyone into BDD, that will be instantly familiar:

Given a context
When an event happens
Then an outcome should occur.

But what about those capabilities which can’t easily be expressed in that easy scenario form?

  • The capability to support peak time loads.
  • The capability to prevent SQL injection attacks.
  • The capability to be easily changed.

Those are, of course, elements of performance, security and quality, also known as non-functionals.

A system which requires these continuous capabilities must always have them, no matter how much it’s changed. There is no particular event which triggers their outcome; no context in which their behavior is exercised, save that the system is operational (or in the case of code quality, merely existent).

Because of that, it’s harder to test that they’ve been delivered. Any test needs to be carried out regularly, as any changes to the system stand a chance of changing the outcome, and often the result of the test isn’t a pass or fail, but a comparison which notifies at some particular threshold. If the system has become less performant, we can always make judgement calls about whether to release it or not. Become a bit hard to change? Oh, well. Slightly less performant than we were hoping for? Let’s fix it later. Open to SQL injection attacks? Um… okay, that’s slightly different. Let’s come back to that one later, too.

There’s a name for a test that we perform regularly.

It’s called monitoring.

The capability to monitor something is discrete.

While it’s hard to describe the actual capabilities, most monitoring scenarios can be expressed easily. Let’s have a conversation with my pixie, and see what happens.

Thistle: So you want this system to be performant, right?

Me: Yep.

Thistle: And we’re going to monitor the performance.

Me: Yep.

Thistle: What do you want the monitor to do?

Me: I want it to tell me if the system stops being able to support the peak load.

Thistle: Okay. Give me an example.

Me: Well, let’s say our peak load is 10 page impressions / second, and we want to be able to serve that in less than 1 second per page. I don’t know what the actual figures are but we’re finding out, so this is just an example…

Thistle: That’s fine; I wanted an example.

Me: Okay. So if the load starts ramping up, and it takes longer than 1 second to serve a page, I want to be notified.

Thistle: So if I write this as a scenario… scuse me, I’m just going to change your three “ifs” to “givens” and a “when”, and add your silent “then”. So it looks like this:

Given a peak load of 10 pages per second
And a threshold of 1 second
When it takes longer than 1 second to serve a page
Then Liz should be notified.

Me:  Yep.

Thistle: Do you want this just in production, or do you need to know about the performance while we’re writing it too?

Me: Well, I need to know about it in production, because then we’ll turn off some of the ads, but that will make us less money so ideally this will never happen. You guys should make sure it supports peak load before it goes live.

Thistle: Okay, so we can use overnight performance tests as a way of monitoring our changes in development. For yours, though, if we’re already sure we’re going to deliver something that meets your peak load requirements, why do we need to worry?

Me: Well, I might end up with more than the peak loads we had before. If the site is really successful, for instance.

Thistle: Ah, okay. So you want to be notified if the page load time rises, regardless of load?

Me: Yep.

Thistle: So your monitoring tool looks like:

Given the system is running
When it takes longer than 1 second to serve a page
Then Liz should be notifed.

Me: That would work.

Thistle: Fantastic.

Me: Are you going to make this magically work, then?

Thistle: Will you pay me in kittens?

Me: What? No!

Thistle: Probably best to get the dev team to do it.

Some things are hard to monitor.

There are some aspects of software which are really hard to monitor. Normally these are the kind of things which result in chaos if the capability is negated, though that’s a symptom rather than a cause of the difficulty. Security is an obvious one, but hardware reliability and data integrity spring to mind. And remember that time Google started storing what the Google Car heard over open wifi networks? Even “being legal” is a capability!

When a capability is hard to monitor, here are my basic guidelines:

  • Learn from other people’s mistakes
  • Add constraints
  • Add safety nets
  • Monitor proxy measurements.

Learning from other people’s mistakes means using industry standards. Persistence libraries are already secured against SQL attacks. Password libraries will salt and hash and encrypt authentication for you.

Constraints might include ensuring that nobody has permission for anything unless it’s explicitly given, refusing service to a particular IP address, or in the worst case scenario, shutting down a server.

Safety nets might include backups, rollback ability, or redundancy like RAID arrays. Reddit’s cute little error aliens ameliorate the effects of performance problems by making us smile anyway (and then hit F5 repeatedly). Being in development is safe; best to fail then, if we can find any problems, or bring experts in who can help us to do so.

Proxy measurements include things like port monitoring (you don’t know what someone’s doing, but you know they shouldn’t be trying to do it there), CRC complexity (it doesn’t tell you how easy it is to change the code, but it’s strongly correlated), bug counts, or the heartbeat from microservices (thanks, Dan and Fred George).

Unfortunately, there will always be instances where we don’t know what we don’t know. If a capability is hard to monitor, you can expect that at some point it will fail, especially if there are nefarious individuals out there determined to make it happen.

It’s worth spending a bit of time to stop them, I think.

Complexity estimation still applies.

The numbers to which I’m referring are explored in more detail over here.

If the capability is one that’s commonly delivered, most of the time you won’t need to worry about it too much. Ops are used to handling permissions. Devs are used to protecting against SQL attacks. Websites these days are generally secure. Use the libraries, the packages, the standard installations, and it should Just Work. (These are 1s and 2s, where everyone or someone in the team knows how to do it). Of course, the usual rules about complacency tipping from simplicity into chaos still apply too.

For capabilities which are particular to industries, domain expertise may still be required. Get the experts in. Have the conversations. (These are 3s, where someone in your company knows how to do it – or you can get that person into your company).

For any capability which is new, try it out. If it’s discrete, look for anything else that might break. If it’s continuous, make sure you try out your monitoring. And look for anything else that might break! (These are the 4s and 5s, where nobody in the world has done it, or someone has, but it was probably your competitor, and you have no idea how they did it.) It’s rare that a continuous capability is a differentiator, but it does sometimes happen.

Above all, and especially for 3s to 5s, make sure that you know who the stakeholder is. For the 4s and 5s it should be easy; it will be the person who championed the project, and your biggest challenge will be explaining the continuous capability in terms of its business benefits. (The number of times I’ve seen legacy replacement systems, intended to be easy to change, lose sight of that vision as sadlines approach…) For 3s, though, the stakeholders are often left out, as if somehow continuous capabilities will be magically delivered. Often they’ve never been happy. And making them happy will be new, so it might end up being a project in and of itself.

That’s mostly because continuous capabilities are difficult to test, so a lot of managers cross their fingers and hope that somehow the team will take these nebulous concerns into account. But now you know.

If you can’t test it, monitor it.

As best you can.

Posted in bdd, capability red, stakeholders, testing | 6 Comments

Using BDD with Legacy Systems

One question I keep being asked is, “Can we use BDD with our legacy systems?”

To help answer this, let me give my simplest definition of BDD:

BDD is the art of using examples in conversation to illustrate behaviour.

So, if you want to talk through the behaviour of your system, and use examples to illustrate that behaviour, yes, you can do this.

There are a couple of benefits of this which might be harder to achieve. The biggest is that using examples in conversation helps you explore behaviour, rather than just specifying it. Examples are easy to discuss, but it’s also easy to decide that they don’t matter, or that you can worry about that scenario later, or that you want different behaviour. It’s harder to do that when you’re talking about tests or specifications, and it’s harder still when the behaviour you’re discussing already exists. If you do find yourself talking through the different examples, you’re probably clarifying behaviour. That’s OK; just recognize that you’re doing it, so some of the usual advice we all give around BDD, and particularly around scope management with BDD, won’t apply. Asking questions, though, is still a great idea. You might find all those places where the behaviour is wrong, or annoying, or isn’t needed at all. Using examples is a great way to illustrate bugs, and helps testers out a lot.

The second aspect is the automation. (If you do this, please consider the other post I wrote about things I like to see in place first; these still apply.) Automation is usually harder with legacy systems, because often they weren’t designed with automation in mind. Websites have elements with no identifiers or meaningful classes, windows applications have complicated pieces with no automation peers, some Adobe widgets assume that if you’re using automation it must be because of reading disabilities so it will helpfully pop up boxes on your screen to ‘help’ you (thank you for that, Adobe).

But the real reason why BDD becomes hard with legacy systems is because often, the system was designed without talking through the behaviour, and the behaviour itself makes no sense.

I recently tried to retrofit SpecFlow around my little toy pet shop. The pet shop itself was designed just as a way of showcasing different automation elements, so it wasn’t particularly realistic. Because of that, I find it impossible now to have conversations about its behaviour, because its behaviour simply isn’t useful. It isn’t how I would design it if I were actually designing a UI for pet shop software. I can’t even talk to my rubber duck about it. I won’t be able to sensibly fit SpecFlow to this until I can actually change the behaviour to something sensible.

If you’re in one of those unfortunate environments with a bit of a blame culture, BDD will help introduce transparency into the quality of your process – or lack of it. Just so you’re warned. (In my instance it was a sensible trade-off at the time, since I originally wanted automation software, not a pet shop, and it’s my software so it’s my problem. You may not be so lucky.)

Automation on legacy systems can give you a nice safety net for any other changes, so it might be worth trying this for a few key scenarios. Teams and particularly testers I’ve worked with have been saved a lot of time in the past by just having one scenario that makes sure the app can start – automation is particularly useful if your build system is closer to your production system than your development one; frequently the case for legacy systems.

If you do happen to find an aspect of behaviour that you like and want to capture, then by all means, do BDD. Talk through the examples, stick them in a wiki, automate them if you can, remembering that having conversations is more important than capturing conversations is more important than automating conversations. You might even find out why things behave a certain way, and come to like the existing behaviour better.

Otherwise, you might want to wait until you’re changing the behaviour to something you like.

Posted in bdd | 12 Comments

The Hobgoblin Manifesto

For Chris, the most Puckish person I know and who I am proud to call my friend.

A Hobgoblin is mischevous from birth
We never mean to harm, but to disrupt,
To muddle up the stasis of the Earth
That others may solve problems they gave up.

A Hobgoblin defies all expectations
Of ugliness and meanness, seeming fair
Creating mirth and joyful celebrations
Whatever else may linger here and there.

A Hobgoblin lives moments day to day
And unintended consequences show.
We own our actions, learn, then seek a way
To make amends, if others will allow.

A Hobgoblin loves fools, and gives them ears
If jackasses they are, and such ears fit
Remembering that when enchantment clears
We’re still fools too – at least a little bit.

A Hobgoblin forgives. One silly act
Does not a person’s whole or part define.
We offer peace, if others will retract;
To err is human; to forgive divine.

A few people have asked if they can sign this! In the spirit of the Community of Thinkers, please post it to your own blog and attrib this if you want these words to be yours too. Derivatives and responses also welcomed.

Posted in Uncategorized | 3 Comments

An open letter to Bob Marshall

Dear Bob,

We’ve had our differences, mostly around a particular set of tweets you sent
out about the Pask award in 2010. I was respectful enough to address those
with you face-to-face. You refused to retract them, particularly the most
hurtful comment, and you told me at the time that I was “naive” for believing
that I won the Pask award fairly.

I told you that it would make me less likely to want to work with you, and you
said you were sorry to hear that. That was as close as I ever got to an

I didn’t want it to get in the way of learning, so I left it aside and reached
past it. I don’t believe that any one aspect of a person affects or defines
all aspects. I attended your RightShifting conference. I offered you some tips
from my experience in Dreyfus to help with your Marshall Model. I was as
forgiving as I feel I could be, and our community interactions seemed good.

I thought that things had become more positive. You reached out and offered me
a hug last time we met, which I reciprocated. I thought that perhaps you had
started to see that maybe I really did deserve that award; that my work was
meaningful. I had dreams that you might one day apologize.

Then, on Friday, this Twitter sequence happened.

@flowchainsensei: I’d like to work on advancing the theory and practice of software and product development. Can’t see that ever happening in the UK.

@somesheep: @flowchainsensei IME @tastapod @lunivore @PapaChrisMatts for example prove you wrong. You can make advancements. Not the ones you want?

‏@flowchainsensei: @somesheep Wise up, dude.

@PapaChrisMatts: How is this tweet consistent with #NVC &
#antimatterprinciple? Cc @somesheep @tastapod @lunivore

@flowchainsensei: @PapaChrisMatts Hobgoblins, little minds, etc. @somesheep @tastapod @lunivore

Bob, did you seriously just resort to name-calling? Because this Twitter sequence makes it look as if you did, and as if you’re doing it to Dan and I particularly. I also found that you appear to have blocked me on Twitter, with – as far as I can tell – zero precursor on my side to deserve it.

Oh, well.

I looked up the word “Hobgoblin” and found some very positive connotations. I
once played Puck, the most famous hobgoblin, as a child, and remembered that
while he’s mischevous, he’s not all bad.

Those that Hobgoblin call you, and sweet Puck,
You do their work, and they shall have good luck;
Are you not he?

So I am a Hobgoblin. I own that, and will as always seek to expand my “little
mind”. I take your insult and make it mine, and wish you good luck in return.
I will turn it round, and use it to create a light-hearted manifesto designed
to steer the mischief we all end up causing in a positive direction, and to
forgive others who cause it themselves.

Gentles, do not reprehend,
If you pardon, we will mend.
And, as I am an honest Puck,
If we have unearned luck
Now to ‘scape the serpent’s tongue,
We will make amends ere long;
Else the Puck a liar call;
So, good night unto you all.
Give me your hands, if we be friends,
And Robin shall restore amends.

If you want to meet up, have a drink, and talk about whatever the hell is
bugging you, I’m more than happy to do so. Just let me know how to get in
touch and make plans, since I can’t DM you any more.


PS: I also apologise for the small jibe I made myself. It was petty and
unnecessary. Sorry.

PPS: I’ve also been pointed to the quote, “A foolish consistency is the hobgoblin of little minds,” which clarifies a lot but not everything. I did ask for clarification over Twitter but of course you probably didn’t see it! Either way, I am happy to join Chris in requesting that if you’re going to preach non-violent communication, please do more of it.

Posted in Uncategorized | 6 Comments