I’ve been invited to speak at the Lean Software and Systems Conference in Atlanta, April 21st to 23rd 2010.
BDD, often described as “TDD done well”, is strongly aligned with Lean
principles. It’s a pull-based approach to delivering software that
matters through the whole software lifecycle, from vision to code. It
minimises rework and over-delivery, bakes quality in, and is founded
on respect for people in their various roles and the language that we
use to collaborate and communicate. BDD has its origins in NLP and the
Sapir-Whorf hypothesis: the idea that the words we use affect the
cognitive model of our work. It decouples the learning associated with
TDD and Acceptance Testing from the word “test”, using the more
natural vocabulary of examples and behaviour to elicit requirements
and create a shared understanding of the domain.
In this talk Liz will introduce BDD, show how to model it using PDCA,
and talk about how its practices can provide quicker feedback and help
us to turn a vision into reality faster.
I’ve been working on this talk for a few months, and hope you enjoy it.
This has been a very special year for me. I’ve been fascinated by Lean principles for a while, but only recently had the opportunity to do some serious study into them and try out the related practices and tools. It’s also taken me some time to work out how strongly aligned Lean Thinking and BDD are. The community around the Lean and Kanban movement has been very welcoming and respectful this year, and I’d urge anyone who is interested to make their way over to the Limited WIP Society, where some excellent blogs, articles and examples of Kanban boards etc. have been collected for your accelerated learning.
We held a bag-stuffing event at Agile 2009, and used it as an exercise in applying Lean techniques. (Hugo Corbucci and Hamlet D’Arcy both posted excellent write-ups of this). One of my proudest moments this year was when some of the Limited WIP and Kanban community asked me to stand up afterwards as someone who could be approached with questions about Lean and Kanban. I’m an effective Agile coach, but my knowledge of Lean still felt new to me at that point. To have this kind of validation meant a lot to me.
I appreciate those of you who have helped me in this journey for your extraordinary patience in teaching me and providing feedback on my understanding. You know who you are.
I just read Michael Dubakov’s excellent post on Kanban psychology, and the shift in thinking from the habits we’re taught to a more rational behaviour.
This post made me remember what it was like, being a developer who’s discouraged from sitting and reading blog posts, or books, or learning more about the job, but who can’t find much meaningful work to do, so starts on the next most valuable non-valuable thing. The feeling was very similar to the way I feel when I can’t work out what to do, but it’s lunchtime soon, I know I’m not completely stuffed, there’s a snack machine down the corridor, and I’ve got into the habit of kinesthetically nibbling on things when I have nothing else occupying my attention.
I can say “no”. Here’s to better dietary habits, for both me and my projects. Long may they last.
Sometimes people ask me, “When we’ve gone Agile… when we are fully Lean… what will it look like?”
The only answer I can come up with is this:
Things will be changing. You’ll be in a better place to respond to change. Your people will have a culture of courage and respect, and will seek continuous improvement, feedback and learning.
I don’t know what your process will look like. The Lean and Agile communities have some ideas you can use to start with. Not all of them will work. Your processes will change, and keep changing.
I have no idea what skills your people will need. The people you have are good people; start with them. The need for their skills will change, and keep changing.
I don’t know what language, tools or technologies you’ll be using. Start with something that’s easy to change. Technology will change, and keep changing.
I don’t know which projects will succeed. Start with the most important project, or the most risky, or the one which has the highest cost of delay. Your market, your business and your customers will change, and keep changing.
There is no end-state with Agile or Lean. You’ll be improving, and continue to improve, trying new things out and discarding the ones which don’t work.
If you do find yourself with an end-state, the chances are that you’ve documented your processes somewhere, and are now asking your teams to adhere to them. Either your process is perfect, or you haven’t reached the end-state yet. I’m guessing your process isn’t perfect. Change, and keep changing.
Chris McMahon mentioned my example of a coaching kanban board in the second of his series of posts against Kanban. That it comes across as simple and infantile in Chris’s example is my fault; I only really touched on the “signal” aspect of the kanban board in this conversation, and didn’t go into more detail. So, in this post, I hope to correct that.
Picture a scene. You’re one of five coaches, hired to help transform the IT department of Screwfix. As part of this effort, you’ve set up a lovely story board on which you put the things you’re working on. This is the kind of thing that we had on our boards (all stories in this post are just examples, I can’t remember the exact concerns):
| Waiting | In Progress | Done |
|---|---|---|
| Run story workshop | Coach team A | Dreyfus models for Devs |
| Competition | ||
| Dreyfus models for BAs | ||
| Coach team B | ||
| Coach team C | ||
| Talk to systems team about environments |
Each day, the coaching team met for a daily stand-up around this board. Our goal was to put our little avatars on the boards to show what we were working on, and move the stories (on post-its) to “done” when we had finished. After a couple of weeks, we realised – we weren’t actually using the board.
“The problem is,” Andy said, “half the things that we’re doing aren’t the kind of things we can move to ‘done’. They’re ongoing.”
“Right,” I agreed, “and half of them aren’t even on this board. There’s all kinds of things we’re doing with respect to the competition, coaching individuals, writing workshops, etc., and they’re not even up here. I wonder what we can do differently?”
At the time, I was hearing a lot of buzz around the word “Kanban”, and reading the Poppendieck’s “Lean Software Development”. It occurred to me that we might be able to use it to help us manage our coaching efforts.
So, I drew up a new board, and talked to our head coach, Chris, about it. “I’ve realised that I can cope with about three concerns at a time,” I said. “If you try to get me to worry about a fourth, I’ll promise to do it, and then something else will drop off the radar. It doesn’t matter how much I promise to do four things, realistically I’ll only get three done well at any time. So, I’m going to limit myself to 3.” I talked to the other coaches, and we found our own limits: 3 for each of the other coaches, and 1 for our part-time coach, Antony. So here’s what our new board looked like:
| Backlog | Liz (3) | Coach team A | Coach team B | Competition |
|---|---|---|---|---|
| Dreyfus models for BAs | Helen (3) | Facilitating retros | Coach team C | |
| Run story workshop | Andy (3) | Getting acceptance tests working with team C | Cont. Integration | Teaching QAs to code |
| Coach new coaches | Antony (1) | Teaching QAs to code | ||
| Phase 3 competition | ||||
| Coach team D | ||||
| Sort out version control | ||||
“Why, Helen!” I exclaimed, grinning. “You’ve got a space there.” I took a new post-it and wrote, “GIVE ME WORK!” on it, then stuck it in Helen’s space.
“What? No!” Helen exclaimed. “I can’t possibly take on any more work!”
“Well, either your limit is wrong – it’s OK to only manage two things – or there’s something you’re working on that’s not up there. Team C’s quite big; is that taking up about twice as much time as normal?”
“Not really,” Helen said thoughtfully. “There’s something else I’m working on. Let me think a moment… Ah! Of course, I’m also running the Agile induction course.” She wrote a post-it to replace mine.
“That’s great,” Chris said. “Now I can see what you’re working on, and also what you’re not working on.” We had a chat about some of the items in our backlog, and talked about when we might be able to bring them into play.
The board was much more effective, helping us juggle our workload appropriately, until it came close to the time for me to leave. Suddenly, I found I had more work than I could possibly manage.
So I cornered Chris. “You know how I said if I ended up doing four things, something would drop off the radar?” I asked.
“Yes…”
“Well, turns out it’s my lunch-hour. I’m exhausted; this isn’t sustainable. Can we sort something out?”
“Of course!” Chris said. So we got the coaches around the board to look at what we were all working on.
Chris looked at the various teams, competitions, workshops, technological strategies and other coaching concerns. “I don’t really care too much about these,” he said, removing about ten items from our (overly large) backlog. “And we don’t need to worry about Team B any more – I think they’ve got it. I’d really like you to run the other coaching workshops before you leave though, Liz.”
“Right,” I said, “then someone else needs to take over the TDD training.”
“I can’t,” Helen said, “I’m not technical. I’ve just finished the last Agile induction course, though, so I’ve got space to pick something else up. Andy’s been facilitating the retros; why don’t I take that instead, and then Andy could do the TDD training?”
“That works,” Andy replied. We swapped stories accordingly, moving them into their new places.
“Fantastic,” I smiled. “Anyone fancy some lunch?”
Having the visual aid helped us prioritise our efforts, as well as letting us share and organise our concerns. The most important aspect of this, for me, was the realistic recognition of our limits, which allowed Chris to direct our focus much more effectively. The Kanban board was just a tool for us, not a process or methodology in its own right. Nor did it replace conversation. It just facilitated it, and acted as a visual radiator. I found it useful, and I hope you enjoyed reading about it.
Andy, the release manager at my current client, asked, “How can I get the teams to follow a consistent process where readying the release for deployment is concerned? We have different version control systems, different build processes… how do I make them behave in the same way?”
Ask for consistent outputs, not consistent processes
In a waterfall-ish world, teams hand over the artifacts of their processes to other teams, who take them away and use them. Each team rarely has insight into what the other teams get up to with their artifacts. The team will produce the artifacts whether they’re needed or not, because they’re part of the process. Release management’s artifact has usually consisted of a document detailing the steps to check out, build and deploy an application, signed off by a gatekeeper who may or may not be familiar with the process itself.
In our lean world, the processes which the team goes through are defined by the team themselves; adapted and changed as the team continually improve. As part of that continual improvement the team may discover new and different ways to collaborate with their stakeholders, including the incidental stakeholders, who may not be paying for the project or using it, but who care nontheless.
Andy is one of these incidental stakeholders.
“If you’re a stakeholder,” I suggest, “you get to put your requirements to the team as well. Ask them for the thing you need. If you need consistency, tell them what consistency looks like. Get them to put the artifacts in a place where you can find them, together with a script that allows you to deploy. Your team are the ones with expertise in deployment, so they might usefully collaborate with the development teams to produce those scripts.” We talk a little about using the Feature Injection template for technical stories, continuous integration, and how the build might play into deployment. “Imagine if you could then just press a button and have those artifacts deploy themselves. How much would that benefit the QAs?”
“Right! And we’d get round the problem we have now, where we don’t deploy the services together with the applications until System Test, where of course they don’t work together.”
“They still won’t work together,” I reply. “All the mistakes in other processes also happen in Lean and Agile; the only difference is that we expect them. We don’t hope they won’t happen – instead, we know they will. We make it safe to fail, instead of fail-safe. This time, we’ll find out while the mistakes are still small, and after that every fix we need will also be small. We’ll also find the mistakes close to the place where they’re created, and the knowledge of how to fix them will be fresh.” We talk about Work In Progress as a liability, and move from there onto the similarities between Lean software development and Lean manufacturing.
Software development is not a production line
A large number of the metaphors of Lean manufacturing which we bring into software development have to do with the production line – that conveyor belt with the “stop the line” Jidoka handles on which the cars are built. Unfortunately, software development isn’t like building cars. If we find ourselves building the same software over and over again, perhaps configured slightly differently depending on the customer, then we’re not creating anything new and we should just use the same software we designed before, or buy something that someone else wrote.
Software development is about creating new things. It’s more like the product development shop; designing new cars, instead of building the same models over and over again. Ideally, it’s about trying out risky things – things which no one has done before. When we try new things, they sometimes don’t work.
“Yes,” Andy comments. “I’d love us to learn better from our mistakes.”
“They’re not really mistakes,” I hazard. “If we’re doing things that nobody has done before, then of course some of them won’t work. If someone is making mistakes it’s either because they’re human – and that isn’t going to change in a hurry – or because the system is set up in such a way that it was always going to happen. We can build safety in to catch the human mistakes. If we blame people for getting things wrong then they have a tendency to hide their mistakes. Instead, we want them to try things out. We want them to learn.”
“We should focus on the good side of making mistakes,” Andy muses.
“Even the act of making mistakes might be considered a good thing. We’re trying things out and learning from them; that’s innovation at work. If we change our language, people behave differently. We can focus on exploring ideas instead of avoiding risk.”
Integration and deployment is a production line
Once we’ve developed some software, we want to get it out into the world. This is where the metaphors of the Lean manufacturing production line come in – Jidoka, or the act of “stopping the line” to fix any problems with it; the button-click from a tester or release manager that signals the need for a fresh deployment, just like Kanban; delivery to the end-customer which provides the feedback to the team.
“Right now, we’ve started to introduce good practices into the team but we’re still not doing automated deployment,” I explain. “That’s like having a high-tech development shop designing the latest cars, then putting them together by hand, using techniques from 1900 – filing this screw here, hammering this panel to fit into place, every time a different set of problems to adapt to.”
“You’re right. I hadn’t thought of it like that,” Andy smiled. “This is going to make things a lot easier.”
Automation doesn’t make it easier
Automation introduces its own set of problems. In my opinion, these are much more interesting problems than, “How do we get our software to work in production?” Instead, we have, “How do we configure our webservers remotely? Do we want virtual servers? How big a sample of production data do we need for UAT? How can we get faster feedback from the build? Can we create a preview site?”
As well as freeing people up to answer these questions, automation allows the release team – rapidly becoming more of a community than a team, and sharing their expertise as they go – to collaborate with the developers, creating software that’s easier to build and deploy, or even coming up with new build tools. The ease of deployment may not increase, but the quality and consistency will.
There are plenty of different types of conveyor belt available – my favourite software conveyor belt right now is Hudson – and they can be configured to twist and turn the different artifacts in all the usual ways. Giving the developers a miniature version – maybe with a smaller data set, or enough to just build their engine or gearbox or whatever module they’re working on – can help to ensure that the software is buildable.
Expect, however, that if you’re doing tricky things with your software that require a unique approach to building it, you may have to spend time designing and building new production line tools.
That’s all right, because they’re just software too.
My first ever article, “Pulling Power: a new Software Lifespan” is up on InfoQ. BDD, Feature Injection, Lean and Kanban playing nice together!
Big thanks to Dan North, Chris Matts, David Anderson, Amr Elssamadisy and the amalgam of developers who make up “Jerry”; also to the Thoughtworkers who reviewed my article and gave me advice, and the Kanbanjins like Eric Willeke who patiently talked me through Lean and Kanban. Several times. Anything I still haven’t understood is my error, not theirs.
I wish I could have put in everything I’ve learnt about Kanban, which is far larger than this article allows. It strikes me as a lovely, simple, high-discipline, least-wasteful way to deliver software, and it matches my feeling that you should fit your process to reality, not reality to your process.
My experiments with Kanban boards so far have been highly successful. Now I want to know about the problems, too. Have you tried Kanban? What happened? Did you try to introduce it? Did that work out? If not, why not? Did you think about introducing it, but decide not to? Tell me your stories!
Simon Baker has written a superb post, capturing the elusive nature of empowerment.
He mentions “an environment that tolerates mistakes to cultivate learning”. For me, this is an essential part of innovation; of the Lean metaphor that suggests a Software Development team is more like a Product Development team (designing new cars) than a Production Line (churning out the same thing repeatedly).
In a Production Line, we try to minimise variance (think Jidoka and Continuous Integration and Deployment).
In a Product Development team, we try to maximise learning opportunities, which means experimenting (think risk, innovation and doing things no one’s done before).
“There is no such thing as a failed experiment, only experiments with unexpected outcomes.”
— Richard Buckminster Fuller
Are mistakes in Software Development really mistakes? Or just experiments from which we learn something new? And if they keep happening… what didn’t we learn?
Comments