The Lean software production line

Andy, the release manager at my current client, asked, “How can I get the teams to follow a consistent process where readying the release for deployment is concerned? We have different version control systems, different build processes… how do I make them behave in the same way?”

Ask for consistent outputs, not consistent processes

In a waterfall-ish world, teams hand over the artifacts of their processes to other teams, who take them away and use them. Each team rarely has insight into what the other teams get up to with their artifacts. The team will produce the artifacts whether they’re needed or not, because they’re part of the process. Release management’s artifact has usually consisted of a document detailing the steps to check out, build and deploy an application, signed off by a gatekeeper who may or may not be familiar with the process itself.

In our lean world, the processes which the team goes through are defined by the team themselves; adapted and changed as the team continually improve. As part of that continual improvement the team may discover new and different ways to collaborate with their stakeholders, including the incidental stakeholders, who may not be paying for the project or using it, but who care nontheless.

Andy is one of these incidental stakeholders.

“If you’re a stakeholder,” I suggest, “you get to put your requirements to the team as well. Ask them for the thing you need. If you need consistency, tell them what consistency looks like. Get them to put the artifacts in a place where you can find them, together with a script that allows you to deploy. Your team are the ones with expertise in deployment, so they might usefully collaborate with the development teams to produce those scripts.” We talk a little about using the Feature Injection template for technical stories, continuous integration, and how the build might play into deployment. “Imagine if you could then just press a button and have those artifacts deploy themselves. How much would that benefit the QAs?”

“Right! And we’d get round the problem we have now, where we don’t deploy the services together with the applications until System Test, where of course they don’t work together.”

“They still won’t work together,” I reply. “All the mistakes in other processes also happen in Lean and Agile; the only difference is that we expect them. We don’t hope they won’t happen – instead, we know they will. We make it safe to fail, instead of fail-safe. This time, we’ll find out while the mistakes are still small, and after that every fix we need will also be small. We’ll also find the mistakes close to the place where they’re created, and the knowledge of how to fix them will be fresh.” We talk about Work In Progress as a liability, and move from there onto the similarities between Lean software development and Lean manufacturing.

Software development is not a production line

A large number of the metaphors of Lean manufacturing which we bring into software development have to do with the production line – that conveyor belt with the “stop the line” Jidoka handles on which the cars are built. Unfortunately, software development isn’t like building cars. If we find ourselves building the same software over and over again, perhaps configured slightly differently depending on the customer, then we’re not creating anything new and we should just use the same software we designed before, or buy something that someone else wrote.

Software development is about creating new things. It’s more like the product development shop; designing new cars, instead of building the same models over and over again. Ideally, it’s about trying out risky things – things which no one has done before. When we try new things, they sometimes don’t work.

“Yes,” Andy comments. “I’d love us to learn better from our mistakes.”

“They’re not really mistakes,” I hazard. “If we’re doing things that nobody has done before, then of course some of them won’t work. If someone is making mistakes it’s either because they’re human – and that isn’t going to change in a hurry – or because the system is set up in such a way that it was always going to happen. We can build safety in to catch the human mistakes. If we blame people for getting things wrong then they have a tendency to hide their mistakes. Instead, we want them to try things out. We want them to learn.”

“We should focus on the good side of making mistakes,” Andy muses.

“Even the act of making mistakes might be considered a good thing. We’re trying things out and learning from them; that’s innovation at work. If we change our language, people behave differently. We can focus on exploring ideas instead of avoiding risk.”

Integration and deployment is a production line

Once we’ve developed some software, we want to get it out into the world. This is where the metaphors of the Lean manufacturing production line come in – Jidoka, or the act of “stopping the line” to fix any problems with it; the button-click from a tester or release manager that signals the need for a fresh deployment, just like Kanban; delivery to the end-customer which provides the feedback to the team.

“Right now, we’ve started to introduce good practices into the team but we’re still not doing automated deployment,” I explain. “That’s like having a high-tech development shop designing the latest cars, then putting them together by hand, using techniques from 1900 – filing this screw here, hammering this panel to fit into place, every time a different set of problems to adapt to.”

“You’re right. I hadn’t thought of it like that,” Andy smiled. “This is going to make things a lot easier.”

Automation doesn’t make it easier

Automation introduces its own set of problems. In my opinion, these are much more interesting problems than, “How do we get our software to work in production?” Instead, we have, “How do we configure our webservers remotely? Do we want virtual servers? How big a sample of production data do we need for UAT? How can we get faster feedback from the build? Can we create a preview site?”

As well as freeing people up to answer these questions, automation allows the release team – rapidly becoming more of a community than a team, and sharing their expertise as they go – to collaborate with the developers, creating software that’s easier to build and deploy, or even coming up with new build tools. The ease of deployment may not increase, but the quality and consistency will.

There are plenty of different types of conveyor belt available – my favourite software conveyor belt right now is Hudson – and they can be configured to twist and turn the different artifacts in all the usual ways. Giving the developers a miniature version – maybe with a smaller data set, or enough to just build their engine or gearbox or whatever module they’re working on – can help to ensure that the software is buildable.

Expect, however, that if you’re doing tricky things with your software that require a unique approach to building it, you may have to spend time designing and building new production line tools.

That’s all right, because they’re just software too.

This entry was posted in kanban, lean. Bookmark the permalink.

One Response to The Lean software production line

  1. I think you nailed it. Writing software is a creative process. Building and deploying it may not be. Steve Traugott pointed out sometime in the 90’s that we build infrastructure like bespoke cars before the advent of the production line. If you’re doing that AND you build software with rivets and a file, you’re doubly challenged.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s