The Oatmeal has a fantastic comic on the Backfire Effect that was so popular, they rewrote it with, um, cleaner language. It starts simply. “You’re not going to believe what I’m about to tell you.” And it goes on to talk about the emotional response that we sometimes have to people saying things that we just don’t want to hear.
I had that reaction when Dave Snowden said, “Agile manages out innovation.”
“What? Of course we don’t.” From anyone else I would have been a bit more defensive, but given Dave’s history of being right about things, I settled for just thinking he was probably wrong this time.
I thought it was just about the parallel probes.
When you’re dealing with truly complex problems, there’s no guarantee that any one experiment will work; so carrying out a bunch of them, all at the same time, is actually more useful than doing them sequentially. Complex, cultural problems aren’t physics; the landscape is going to change anyway, and eliminating side-effects isn’t as important as finding ones which happen to work in your favour.
Of course, most of us aren’t running experiments in parallel. We have one product we’re working on, and we’re iterating towards a final solution, knowing vaguely what we want to achieve and getting feedback along the way. It’s very rare that we don’t have some idea of what we ought to be doing.
Agile doesn’t actually prevent us from running parallel probes, though. I once worked in a mixed dev/coach role writing a new trading platform. It wasn’t entirely an Agile project. They’d brought in some UI designers before-hand, who had carefully researched, then created, the design of the UI they wanted. Back then I already knew that if it was new, it was likely to be full of discoveries, so I hard-coded it and got it in front of the traders, who hated it.
Of course, being not entirely an Agile project, our UI designers had gone on to pastures new, so we were left with the terrible position of having to let the developers come up with the design.
Three of us, independently, came up with something we thought might work. The traders quite liked two of the solutions (or at least, they talked about what was wrong with them rather than why they were wrong, which was much better), so we kept those and made them work together, iterating to fix everything. But we did work in parallel for a bit, and it fitted nicely into our Sprint.
I couldn’t help feeling that I’d missed something, though. Dave didn’t just say that we tend not to be innovative; he said we actively manage out innovation. It took me a while to work out what was wrong.
It’s our focus on customers that holds us back.
Stop. Hold on. What? How can that possibly be the wrong thing to do?
It’s such an important part of Agile that it’s right there in the first principle:
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
– The Agile Manifesto
The Kanban body of knowledge isn’t any better, either; all the Lean principles from which it derives are also very customer-focused, with the value stream being measured from when the customer need is discovered to when value is realized.
But how could that possibly be the wrong thing to do?
The most innovative companies aren’t just satisfying their existing customers.
Amazon doesn’t just sell books. Google doesn’t just do search. Netflix doesn’t just stream content. John Menzies, once the darling newsagent of high streets, now does airport logistics. Nintendo once made playing cards. Even Toyota started as a loom manufacturer.
At smaller scales, huge number of inventions are no longer driven by their original intent. Kleenex was originally made to take off make-up. The Slinky was meant for stabilizing ship instruments. Microwaves were discovered while messing with radar. And a monk called Dom Pierre Perignon was once tasked with removing the unwanted fizz from the wine.
All of these have something in common: they repurposed something they already had.
This is exaptation; the re-use of things for purposes for which they were not designed or evolved.
A lot of these discoveries and inventions have happened by chance… but what if there was a way to deliberately cause this kind of innovation?
This is what Dave calls “Managing for Serendipity”.
I’m pretty new to the entire field, so worth looking for his material on it rather than trusting me. In basic terms, though, it means looking to repurpose what you already have, to satisfy new varieties of customers.
Cognitive Edge use Sensemaker surveys to look for patterns in customer need and existing technology, but even introducing different kinds of experts from outside of your core domain to what you do can help to generate new ideas (these are the “oblique” probes in your probe portfolio).
IBM repurposed sewing machine punch-card technology for use in computing. Amazon repurposed their back-end and sold it as a product; AWS. Edin Berliner repurposed Edison’s telephone message recorder and invented the music industry. So many inventions are there, waiting to be discovered… unless we fall into that “first-fit pattern-matching” trap, and focus only on the one customer we already have.
“Innovation” comes from the Latin “to introduce as new”. If you’re being truly innovative, it’s the customers that are new, too.
Liz Keogh, lunivore 
Lovely post Liz.
Agile comes from the Continuous Improvement stable doesn’t it?
And there has long been a view that lots of Continuous Improvement won’t add up to strategic change or advance. Even though Agile does specifically look for emergence, it’s bounded emergence.
So this seems like a new take on an older problem. Systems thinking is about finding the right intervention in the system to meet a declared or emergent purpose . Agile is a good way of delivering those interventions without crushing the original objective. They can work together much better than the old Strategy vs Delivery dichotomy.
But important to know where the boundaries are
Thanks Kathryn!
I’m still a fan of continuous improvement – repurposing something doesn’t have to be a big step at first! It’s just that Agile is continuously improving in a fixed direction, with limited deviation; your “bounded emergence” here. It doesn’t have to be so bounded to be continuous.
I really like feathers as an example of exaptation. They started probably as insulation for dinosaurs; moved to a signalling mechanism, and eventually allowed dinosaurs to glide, then fly. Each change was small, but the end result was an exaptation of the original evolutionary advantage. There were probably some phase shifts where the advantage was suddenly really great and it took off. Dave’s recent posts on Flexuosity / Apex Predator Theory are well worth looking at from that perspective.
Re boundaries: Dave does a lot of work around constraints; I’ve written a couple of posts on them. Enabling constraints vs. governing constraints is really interesting. And re strategy: I’ve got another blog post I’m working on (hence the lack of posts lately, it’s taking a while!) and Wardley Mapping is well worth looking at too.
This is really disappointing. I think to understand how “Agile” really views innovation it’s important to understand that the manifesto for Agile software development is not the definition of Agile in general, which happened ten years earlier (and btw has nothing to do with continuous improvement).
Thanks to people like Rick Dove, we’ve known since the ’90s that an Agile architecture is defined as one capable of being re-configured to new and unforeseen purpose. Also that Agile thinking includes the innovation system feedback loop which (done well) would reveal the potential in the new use.
Which doesn’t make the article unhelpful or less interesting – it just ignores a substantial body of knowledge and experience.
Mike, I was a child for most of the 90s so sorry to be missing your extensive expertise here. Really happy to read up on it if you’re willing to share links. As I understood it, the name “Agile” for the Software movement was chosen at Snowbird – was that not the case?
If there are different Agile principles that are explicitly positioned as “Agile” but aren’t aligned with the Agile Manifesto then I’d love to know more; this is the first I’ve heard of that.
(I’ve found some of Rick Dove’s work from earlier – I was familiar with his name and didn’t know his work predated the manifesto so grateful for the heads-up – but it seems to be using “Agile” as a descriptor for adaptability alongside Lean techniques, focused on manufacturing and later defense; not as part of the wider Software change movement; it also explictly mentions pull-systems and waste reduction, which would still eliminate variance and serendipity.)
And so sorry to have disappointed you. What would you have preferred to see here?
Liz, first I want to apologise for the tone of that post. I have a long reason that involves logging on to WordPress and losing sight of the draft, but it’s no excuse. Hopefully I can do better now – here’s a post that a more professional version of me might have written
Alas, I’ve increasingly been faced with this sense of Agile as purely solution oriented – without a place for innovation. I find this disappointing, as it wasn’t my experience within the ‘low-ceremony process’ community before Snowbird, and I don’t think this was the intention of that group. What happened?
Personally, what I see in recent years is almost an industry for whom positioning agile as execution-only is important. For these people, the re-definition of agile allows them to pitch their solution into the resulting problem space. “Design Thinking” practitioners are the main culprits, but it’s not an exclusive club.
I know that my simply saying “but that’s not agile” is just nay-saying of the least helpful kind. So to add some evidence, I typically point to the work of the Agile Manufacturing Forum, a group that got together some ten years before Snowbird. (Indeed the term ‘agile’ was evangelised at Snowbird by at least one attendee (Jim Highsmith) familiar with their work.) I like this version of Agile, and I think a lot of what they say is being re-discovered in places now.
The Forum defined agility as including innovation systems. They also specified that agile capability would necessarily include the ability to re-configure and re-purpose technology and organisation to address unforeseen situations. (As distinct from ‘flexibility’ to address multiple foreseen situations). There are more, deeper, insights in their work that I think would interest anyone looking to understand broader agile systems. If anyone would like to read more, Rick Dove, a mainstay of the Agile Manufacturing Forum is still active at parshift.com, and with incose.org. I’ve also put some favourite docs on dropbox @ https://bit.ly/3nwFbMl
Both manufacturing and software versions of Agile claim roots in the 1986 HBR article “The New New Product Development Game”. Amongst other things, this research advocates what they call self-transcendence to “override the status quo and make the big discovery” and cross fertilisation for “diversity (to) foster new ideas and concepts”.
For me, if Agile doesn’t address innovation, it’s because we let it be squeezed out…
Thanks Mike, appreciate the apology and this makes sense. Also appreciate the folder with the links. The fact that so much material with an “Agile brand” existed well before the manifesto was quite eye-opening to me, so thanks for that.
Dave talks a lot about the reconfiguration and repurposing (he uses “recombination” a lot); and I agree that a good Agile architecture does allow for this. Mostly I see this being used to adapt to changing requirements and replacement of systems; maintenance rather than innovation, though I also agree that if it’s done well it allows for both.
I think the difference between Agile and the way that Dave talks explicitly about “Managing for Serendipity” is captured in a couple of words you’ve used here: “unforseen situations”. With all the versions of Agile I’ve seen (and with the greatest of respect, the prior work you’re discussing here seems no different), the need to exapt for different customers or markets is largely accidental.
I haven’t found anything which *deliberately* introduces those unforseen situations. Even in “The New New Product Development Game”, cross-fertilization is used to refer to different disciplines from within technical teams, rather than potentially different customers. The focus is still on iteration towards a vaguely known solution, and this is echoed throughout, for example:
“Canon’s design engineers stayed alert throughout the process to make sure their design was being converted into what they had in mind.”
“Team members engage in a continual process of trial and error to narrow down the number of alternatives that they must consider. ”
Those are very different from some of the techniques Dave has suggested, which are designed to deliberately increase, rather than decrease, variance and possible alternatives. Deliberately introducing experts who might suggest other uses was one such technique; he has others involving deliberate blind mutation of existing systems.
That a good, well-designed, Agile system lends itself to exaptation, as well as easy maintenance… that, I definitely agree with. The *possibility* of exaptation is not managed out, at all. But I think that the *probability* of it happening is lower. To cause exaptation, rather than merely allow for it, requires a deliberate intervention that I have yet to see mentioned in the same place as Agile (I am keeping an eye out though!).
Completely agree with the “Agile manages out innovation” concept. In my view, Agile as I’ve experienced it on several projects is nothing but micro-management on steroids. Also have heard, but not yet fact-checked myself, that all of the authors of the original Agile Manifesto have renounced what Agile has become. Maybe that could be the subject of a future article?
The bastardization of Agile as a concept is definitely another post I think (there are many out there already), but I completely agree.
I’m lucky enough to have experienced really great Agile projects that genuinely delivered value. Even with that version of Agile though, they were very, very focused on delivering value to existing customers only – showcases to those customers every Sprint, for instance; welcoming their feedback and changing requirements; but without introducing anyone else to the domain.
Based on Mike Robinson’s comment above I’ve also been looking through Rick Dove’s work – and even there, the innovation cycle is restricted to the given domain; “Health Domain” for instance. There’s no mention (at least that I’ve found) of the deliberate introduction of external perspective which would be needed for serendipity.
So it’s not just that Agile in its worst form manages out innovation. It does it in its best form, too. The innovation that great Agile projects allow for is limited to the Complicated-Complex liminal threshold; iterating towards a known solution. Realizing that (with Dave’s usual extensive and patient help) was kinda mind-blowing for me.
Pingback: Java Weekly, Issue 364 | Baeldung