Liz Keogh's blog

I’ve been invited to speak at the Lean Software and Systems Conference in Atlanta, April 21st to 23rd 2010.

---

BDD, often described as “TDD done well”, is strongly aligned with Lean
principles. It’s a pull-based approach to delivering software that
matters through the whole software lifecycle, from vision to code. It
minimises rework and over-delivery, bakes quality in, and is founded
on respect for people in their various roles and the language that we
use to collaborate and communicate. BDD has its origins in NLP and the
Sapir-Whorf hypothesis: the idea that the words we use affect the
cognitive model of our work. It decouples the learning associated with
TDD and Acceptance Testing from the word “test”, using the more
natural vocabulary of examples and behaviour to elicit requirements
and create a shared understanding of the domain.

In this talk Liz will introduce BDD, show how to model it using PDCA,
and talk about how its practices can provide quicker feedback and help
us to turn a vision into reality faster.

---

I’ve been working on this talk for a few months, and hope you enjoy it.

This has been a very special year for me. I’ve been fascinated by Lean principles for a while, but only recently had the opportunity to do some serious study into them and try out the related practices and tools. It’s also taken me some time to work out how strongly aligned Lean Thinking and BDD are. The community around the Lean and Kanban movement has been very welcoming and respectful this year, and I’d urge anyone who is interested to make their way over to the Limited WIP Society, where some excellent blogs, articles and examples of Kanban boards etc. have been collected for your accelerated learning.

We held a bag-stuffing event at Agile 2009, and used it as an exercise in applying Lean techniques. (Hugo Corbucci and Hamlet D’Arcy both posted excellent write-ups of this). One of my proudest moments this year was when some of the Limited WIP and Kanban community asked me to stand up afterwards as someone who could be approached with questions about Lean and Kanban. I’m an effective Agile coach, but my knowledge of Lean still felt new to me at that point. To have this kind of validation meant a lot to me.

I appreciate those of you who have helped me in this journey for your extraordinary patience in teaching me and providing feedback on my understanding. You know who you are.

I just read Michael Dubakov’s excellent post on Kanban psychology, and the shift in thinking from the habits we’re taught to a more rational behaviour.

This post made me remember what it was like, being a developer who’s discouraged from sitting and reading blog posts, or books, or learning more about the job, but who can’t find much meaningful work to do, so starts on the next most valuable non-valuable thing. The feeling was very similar to the way I feel when I can’t work out what to do, but it’s lunchtime soon, I know I’m not completely stuffed, there’s a snack machine down the corridor, and I’ve got into the habit of kinesthetically nibbling on things when I have nothing else occupying my attention.

I can say “no”. Here’s to better dietary habits, for both me and my projects. Long may they last.

The paradox of mocking

When we code from the outside-in in BDD, we start with the layer we know – the UI, often graphical – establish collaborators for the UI, establish collaborators for those classes, and work our way inwards until we run out of collaborators we haven’t coded.

We write examples (tests) for each unit of code, all the way down, and we usually express the collaborations with mocks.

The trouble is, we don’t really know how the class we’re about to code is going to use its collaborators. We can only guess. When I actually come to code the class, I often find that I want to use the collaborator in a different way to the way I expected. When I come to code the collaborator, I might find that it needs more information than its consuming class is giving it to do its job properly.

In this case, I have to back up and change the way that I’m expressing the collaboration in the examples. I change method names and signatures for my collaborator according to the things I’ve learnt from actually using it.

I’d rather do this – work out how I actually want to use a class, then change the descriptions – than be forced to conform to the guesswork that I made.

Multi-pair stories

Yesterday, one of the development team said, “If you’re doing XP, you only have one pair on each story, so if you have four developers, you’ll have two stories in play. It doesn’t make any sense to try to limit it to one.”

I’ve heard some of the Kanban community talk about “swarming” a feature; getting a whole team of, say, 8 developers to take it on and complete it in a short time. I’ve also found that some of them prefer not to split up the features as finely as I do, into very thin vertical slices; instead, the teams work on something marketable.

This makes sense, if you’re going to parcel out chunks of code. The idea of slicing things horizontally goes against most of what I teach about how to write stories – and yet, it does allow a larger team to get something valuable out the door more quickly than BDD’s pure outside-in.

It turns out that guessing what might need to happen further down a stack isn’t much different to guessing what happens with a collaborator, before the class is actually written. Realising this has made me revisit my assumptions about the need to do pure outside-in work. So how can we do this and still call it BDD? How can we gain confidence that we’re writing software that really matters, and doing so efficiently?

I can remember occasions at Thoughtworks where we did this – particularly, finishing off features to hit deadlines at the Guardian. Some of the developers at my current client are also working this way, as are others in the industry. So, here are my suggestions for making this work.

BDD for swarms

• Use scenarios. Having a clear understanding of the feature and a set of scenarios to which the team can work to helps keep everyone focused on the actual behaviour needed. This means it’s less likely that pairs and individuals will write code “just in case”. If in doubt, YAGNI. The scenarios will tell you if you missed something, once all the chunks are integrated.
• Get something working. Start with the happy path or simplest scenario, and integrate this as soon as you can. This will provide a “backbone” for the rest of the scenarios. It also lets the team play through the different scenarios to test their work.
• Mock out collaborators. If you need other classes for end-to-end testing, knock up something really simple or use a mock to allow you to get some feedback. For instance, I make a smiley face appear on my Game of Life exercise when I click the button, just so that I can see that the GUI events are wired up. It takes about 5 minutes to do something like this. Name your simple classes in a way which makes it obvious, so that they don’t get mixed up with the production code and can be safely deleted later.
• Talk to the team. Conversation around what you’re doing will be crucial! If an interface has been defined at the boundary of the code, and you need to change it, go tell the other teams. They’ll be able to adapt to, or at least be aware of, the changes.
• The rest of the team is your customer. When it comes to the name of a method or the value that’s wanted from that method, the “upstream” team – closer to the outside – trump the downstream team. When it comes to the information that’s required to do the job, the downstream team win.
  
List pets = petStore.getPets(String name);

  The upstream team can ask to change the List to a Set, or the name of the method to getPetsByName. The downstream team can say, “Hey, we need the type of pet too; we’ve got a hamster, a rabbit and a kitten all called Snowy.”
• Prefer code that’s easy to use over code that’s easy to write. If you’re upstream, trust that the rest of your team will be able to solve the problem you’re about to set for them. Keep pushing the complexity down, and remember: Tell, don’t ask.
• Use examples to drive your code. When you code a module further down the stack without looking to see what’s going to use it, you’re indulging in guesswork. Write some examples of how you expect the consuming class to be using yours, then revisit them when you have more information.
• Write examples to help other people code. Ideally, the upstream team will provide examples of how they want the collaborator to behave to the downstream team. These examples can be built incrementally, as the upstream team come up with features.
• Share code. If the teams check in before the code is finished, their scenarios will fail. If they check in examples which haven’t yet been coded, those examples will fail. This won’t be a problem if no one else is modifying the code base; however, if it’s a subset of a much larger team breaking the build can cause havoc, and the habit of keeping builds green is a good one. Try distributed version control, which will allow a team to check in on USB keys or a temporary space until the code works. (There are techniques for getting, say, Mercurial, to work alongside, say, Subversion – mostly by making each system ignore the other). You could also pass around patch files to keep the code in sync.
• Delete unused code. If at all possible, check the code as it’s being integrated and delete anything which isn’t actually being used or going to be used. This might include things like database columns, layers of abstraction which aren’t actually valuable, etc.
• Look for quick feedback. If you haven’t integrated your code within a few hours, it’s probably wrong. Making the assumption that it’s wrong should help you want to integrate it more frequently.
• The more, the merrier! Look over each other’s shoulders as you pass! There’s nothing wrong with tripling or quadrupling instead of pairing, if it means that a pair is more likely to write the right software later.

Edit:

Eric Willeke responded to my post with his own perspective on swarming. I very much like the idea of having the skeleton (the simplest scenario?) ready in time for the rest of the team to get on board.

Scrum: Hm, you’re rather small. Are you sure you want to do this?

Kanban: Bring it.

Scrum: Right. I represent a fundamental mindshift in the way that people do projects.

Kanban: So do I. My mindshift is different to yours; it feels subtler.

Scrum: I doubt that. I help teams collaborate and deliver working software iteratively.

Kanban: As do I. Unlike you, I don’t tie people down to fixed iterations; I let them find their own cadence.

Scrum: You say that like it’s a good thing! What if people aren’t even used to delivering iteratively? How can you possibly stop novices from taking weeks over their increments of code?

Kanban: Well, maybe I don’t work so well with novice teams. A high-discipline team, though, who can keep their flow consistent, will find themselves more responsive with me than they will with you.

Scrum: I help new teams get started. I give them easy, simple approaches that they can follow. You’re all about the maths.

Kanban: Actually, I’m all about the theory of constraints. By limiting work in progress, I make the bottlenecks obvious.

Scrum: I do that by focusing on getting the work through to production. What’s more, I was designed for software. You’re just the bastard offspring of manufacturing; you have no relevance!

Kanban: If I have no relevance, then why do so many people believe that I’m valuable, and fight to defend me?

Scrum: They’re in it for the money…

Kanban: Says the world’s fastest-growing pyramid scheme…

Scrum: I resent that! There are plenty of successful Scrum projects out there. We’re good people; we just want to change the world.

Kanban: I’ll settle for continuous improvement, thanks.

Scrum: We’ve got that too, except we call them Retrospectives.

Kanban: You don’t need to batch up your improvements if everyone is focused on it.

Scrum: There’s nothing to prevent us from continually improving.

Kanban: There’s nothing to prevent us from having retrospectives, if we need them, or in fact from taking on any of your valuable practices. I’m just like you, except that I have sensible limits.

Scrum: You’re not just like me. You don’t even worry about estimates half the time.

Kanban: We estimate in time to deliver, not story points. The business understand this.

Scrum: The business don’t trust my teams; they haven’t delivered successfully for a while. My planning enables them to regain that trust. You can’t even work without it.

Kanban: That’s true. Once we have the trust, though, we don’t need to waste as much time on planning and estimating as you do.

Scrum: How can you call a practice that establishes trust wasteful?

Kanban: You’re right. We value people too, you know – we work well with the Lean principles, and “Respect for People” is one of the pillars.

Scrum: Well, that’s good to know. People are the most important part of the process.

Kanban: We seem to have quite a lot in common. Thinking about it, you’d probably make quite a good stepping-stone for people learning to deliver software iteratively for the first time. I guess they could turn to me later, once the discipline is there.

Scrum: That might work. I know some people have used you in odd ways, and found value. Maybe we should just be friends?

Kanban: Let’s call it a draw.

XP: Let’s not. You may have the planning and estimation sewn up; you may be shifting your constraints and delivering responsively. Neither of you can survive without my technical practices. Get over here!

FATALITY!

Winner: XP

Chris McMahon mentioned my example of a coaching kanban board in the second of his series of posts against Kanban. That it comes across as simple and infantile in Chris’s example is my fault; I only really touched on the “signal” aspect of the kanban board in this conversation, and didn’t go into more detail. So, in this post, I hope to correct that.

Picture a scene. You’re one of five coaches, hired to help transform the IT department of Screwfix. As part of this effort, you’ve set up a lovely story board on which you put the things you’re working on. This is the kind of thing that we had on our boards (all stories in this post are just examples, I can’t remember the exact concerns):

Waiting	In Progress	Done
Run story workshop	Coach team A	Dreyfus models for Devs
Competition
Dreyfus models for BAs
Coach team B
Coach team C
	Talk to systems team about environments

Each day, the coaching team met for a daily stand-up around this board. Our goal was to put our little avatars on the boards to show what we were working on, and move the stories (on post-its) to “done” when we had finished. After a couple of weeks, we realised – we weren’t actually using the board.

“The problem is,” Andy said, “half the things that we’re doing aren’t the kind of things we can move to ‘done’. They’re ongoing.”

“Right,” I agreed, “and half of them aren’t even on this board. There’s all kinds of things we’re doing with respect to the competition, coaching individuals, writing workshops, etc., and they’re not even up here. I wonder what we can do differently?”

At the time, I was hearing a lot of buzz around the word “Kanban”, and reading the Poppendieck’s “Lean Software Development”. It occurred to me that we might be able to use it to help us manage our coaching efforts.

So, I drew up a new board, and talked to our head coach, Chris, about it. “I’ve realised that I can cope with about three concerns at a time,” I said. “If you try to get me to worry about a fourth, I’ll promise to do it, and then something else will drop off the radar. It doesn’t matter how much I promise to do four things, realistically I’ll only get three done well at any time. So, I’m going to limit myself to 3.” I talked to the other coaches, and we found our own limits: 3 for each of the other coaches, and 1 for our part-time coach, Antony. So here’s what our new board looked like:

Backlog	Liz (3)	Coach team A	Coach team B	Competition
Dreyfus models for BAs	Helen (3)	Facilitating retros	Coach team C
Run story workshop	Andy (3)	Getting acceptance tests working with team C	Cont. Integration	Teaching QAs to code
Coach new coaches	Antony (1)	Teaching QAs to code
Phase 3 competition
Coach team D
Sort out version control

“Why, Helen!” I exclaimed, grinning. “You’ve got a space there.” I took a new post-it and wrote, “GIVE ME WORK!” on it, then stuck it in Helen’s space.

“What? No!” Helen exclaimed. “I can’t possibly take on any more work!”

“Well, either your limit is wrong – it’s OK to only manage two things – or there’s something you’re working on that’s not up there. Team C’s quite big; is that taking up about twice as much time as normal?”

“Not really,” Helen said thoughtfully. “There’s something else I’m working on. Let me think a moment… Ah! Of course, I’m also running the Agile induction course.” She wrote a post-it to replace mine.

“That’s great,” Chris said. “Now I can see what you’re working on, and also what you’re not working on.” We had a chat about some of the items in our backlog, and talked about when we might be able to bring them into play.

The board was much more effective, helping us juggle our workload appropriately, until it came close to the time for me to leave. Suddenly, I found I had more work than I could possibly manage.

So I cornered Chris. “You know how I said if I ended up doing four things, something would drop off the radar?” I asked.

“Yes…”

“Well, turns out it’s my lunch-hour. I’m exhausted; this isn’t sustainable. Can we sort something out?”

“Of course!” Chris said. So we got the coaches around the board to look at what we were all working on.

Chris looked at the various teams, competitions, workshops, technological strategies and other coaching concerns. “I don’t really care too much about these,” he said, removing about ten items from our (overly large) backlog. “And we don’t need to worry about Team B any more – I think they’ve got it. I’d really like you to run the other coaching workshops before you leave though, Liz.”

“Right,” I said, “then someone else needs to take over the TDD training.”

“I can’t,” Helen said, “I’m not technical. I’ve just finished the last Agile induction course, though, so I’ve got space to pick something else up. Andy’s been facilitating the retros; why don’t I take that instead, and then Andy could do the TDD training?”

“That works,” Andy replied. We swapped stories accordingly, moving them into their new places.

“Fantastic,” I smiled. “Anyone fancy some lunch?”

Having the visual aid helped us prioritise our efforts, as well as letting us share and organise our concerns. The most important aspect of this, for me, was the realistic recognition of our limits, which allowed Chris to direct our focus much more effectively. The Kanban board was just a tool for us, not a process or methodology in its own right. Nor did it replace conversation. It just facilitated it, and acted as a visual radiator. I found it useful, and I hope you enjoyed reading about it.

Andy, the release manager at my current client, asked, “How can I get the teams to follow a consistent process where readying the release for deployment is concerned? We have different version control systems, different build processes… how do I make them behave in the same way?”

Ask for consistent outputs, not consistent processes

In a waterfall-ish world, teams hand over the artifacts of their processes to other teams, who take them away and use them. Each team rarely has insight into what the other teams get up to with their artifacts. The team will produce the artifacts whether they’re needed or not, because they’re part of the process. Release management’s artifact has usually consisted of a document detailing the steps to check out, build and deploy an application, signed off by a gatekeeper who may or may not be familiar with the process itself.

In our lean world, the processes which the team goes through are defined by the team themselves; adapted and changed as the team continually improve. As part of that continual improvement the team may discover new and different ways to collaborate with their stakeholders, including the incidental stakeholders, who may not be paying for the project or using it, but who care nontheless.

Andy is one of these incidental stakeholders.

“If you’re a stakeholder,” I suggest, “you get to put your requirements to the team as well. Ask them for the thing you need. If you need consistency, tell them what consistency looks like. Get them to put the artifacts in a place where you can find them, together with a script that allows you to deploy. Your team are the ones with expertise in deployment, so they might usefully collaborate with the development teams to produce those scripts.” We talk a little about using the Feature Injection template for technical stories, continuous integration, and how the build might play into deployment. “Imagine if you could then just press a button and have those artifacts deploy themselves. How much would that benefit the QAs?”

“Right! And we’d get round the problem we have now, where we don’t deploy the services together with the applications until System Test, where of course they don’t work together.”

“They still won’t work together,” I reply. “All the mistakes in other processes also happen in Lean and Agile; the only difference is that we expect them. We don’t hope they won’t happen – instead, we know they will. We make it safe to fail, instead of fail-safe. This time, we’ll find out while the mistakes are still small, and after that every fix we need will also be small. We’ll also find the mistakes close to the place where they’re created, and the knowledge of how to fix them will be fresh.” We talk about Work In Progress as a liability, and move from there onto the similarities between Lean software development and Lean manufacturing.

Software development is not a production line

A large number of the metaphors of Lean manufacturing which we bring into software development have to do with the production line – that conveyor belt with the “stop the line” Jidoka handles on which the cars are built. Unfortunately, software development isn’t like building cars. If we find ourselves building the same software over and over again, perhaps configured slightly differently depending on the customer, then we’re not creating anything new and we should just use the same software we designed before, or buy something that someone else wrote.

Software development is about creating new things. It’s more like the product development shop; designing new cars, instead of building the same models over and over again. Ideally, it’s about trying out risky things – things which no one has done before. When we try new things, they sometimes don’t work.

“Yes,” Andy comments. “I’d love us to learn better from our mistakes.”

“They’re not really mistakes,” I hazard. “If we’re doing things that nobody has done before, then of course some of them won’t work. If someone is making mistakes it’s either because they’re human – and that isn’t going to change in a hurry – or because the system is set up in such a way that it was always going to happen. We can build safety in to catch the human mistakes. If we blame people for getting things wrong then they have a tendency to hide their mistakes. Instead, we want them to try things out. We want them to learn.”

“We should focus on the good side of making mistakes,” Andy muses.

“Even the act of making mistakes might be considered a good thing. We’re trying things out and learning from them; that’s innovation at work. If we change our language, people behave differently. We can focus on exploring ideas instead of avoiding risk.”

Integration and deployment is a production line

Once we’ve developed some software, we want to get it out into the world. This is where the metaphors of the Lean manufacturing production line come in – Jidoka, or the act of “stopping the line” to fix any problems with it; the button-click from a tester or release manager that signals the need for a fresh deployment, just like Kanban; delivery to the end-customer which provides the feedback to the team.

“Right now, we’ve started to introduce good practices into the team but we’re still not doing automated deployment,” I explain. “That’s like having a high-tech development shop designing the latest cars, then putting them together by hand, using techniques from 1900 – filing this screw here, hammering this panel to fit into place, every time a different set of problems to adapt to.”

“You’re right. I hadn’t thought of it like that,” Andy smiled. “This is going to make things a lot easier.”

Automation doesn’t make it easier

Automation introduces its own set of problems. In my opinion, these are much more interesting problems than, “How do we get our software to work in production?” Instead, we have, “How do we configure our webservers remotely? Do we want virtual servers? How big a sample of production data do we need for UAT? How can we get faster feedback from the build? Can we create a preview site?”

As well as freeing people up to answer these questions, automation allows the release team – rapidly becoming more of a community than a team, and sharing their expertise as they go – to collaborate with the developers, creating software that’s easier to build and deploy, or even coming up with new build tools. The ease of deployment may not increase, but the quality and consistency will.

There are plenty of different types of conveyor belt available – my favourite software conveyor belt right now is Hudson – and they can be configured to twist and turn the different artifacts in all the usual ways. Giving the developers a miniature version – maybe with a smaller data set, or enough to just build their engine or gearbox or whatever module they’re working on – can help to ensure that the software is buildable.

Expect, however, that if you’re doing tricky things with your software that require a unique approach to building it, you may have to spend time designing and building new production line tools.

That’s all right, because they’re just software too.

My first ever article, “Pulling Power: a new Software Lifespan” is up on InfoQ. BDD, Feature Injection, Lean and Kanban playing nice together!

Big thanks to Dan North, Chris Matts, David Anderson, Amr Elssamadisy and the amalgam of developers who make up “Jerry”; also to the Thoughtworkers who reviewed my article and gave me advice, and the Kanbanjins like Eric Willeke who patiently talked me through Lean and Kanban. Several times. Anything I still haven’t understood is my error, not theirs.

I wish I could have put in everything I’ve learnt about Kanban, which is far larger than this article allows. It strikes me as a lovely, simple, high-discipline, least-wasteful way to deliver software, and it matches my feeling that you should fit your process to reality, not reality to your process.

My experiments with Kanban boards so far have been highly successful. Now I want to know about the problems, too. Have you tried Kanban? What happened? Did you try to introduce it? Did that work out? If not, why not? Did you think about introducing it, but decide not to? Tell me your stories!