Perverse Incentives

I proposed an Open Space session at Agile Mumbai to discuss Perverse Incentives and gaming the system in Agile projects.

A perverse incentive is one which produces an effect that works against the interests of the people who proposed the incentive in the first place. Gaming the system can be thought of as working deliberately through the path of least resistance or greatest reward, towards the incentive, even if the effect of working that way is perverse.

Many thanks to the Open Spacers who came up with the practices.

Here are some situations which led to the discussion.

  • Situation 1: An IM notices that the team often underestimate cards. He berates the team for not spotting the problems that will slow them down and fixing them or at least factoring them in.
  • Desired effect: The team deal with the problems that are slowing them down and speed up. The estimates become more accurate once the surprises have gone away.
  • Actual effect: The team factor in larger estimates and contingency – it’s easier than spotting and dealing with the tricky problems. Because they don’t always need that contingency, there’s less pressure to finish the tasks to which they’ve committed themselves. Without that pressure, they slow down.
  • Situation 2: Two developers go off on their own to investigate a possible solution to a tricky problem. They don’t tell anyone they’re doing it. The solution they’re thinking of was already investigated and discounted. When the PM realises how much time has been wasted, he puts a process in place to ensure that any work that gets started has been approved with the team lead, that the business value has been agreed with the BAs, and that acceptance criteria have been defined. Anyone who starts work without following the process is reminded of the time wasted on the project by people doing work that’s not valuable.
  • Desired effect: Less time is wasted doing work that doesn’t deliver business value.
  • Actual effect: Developers waste time waiting for the team leads and BAs to come out of meetings. Refactoring and other cleaning up, which doesn’t deliver business value directly, slides. The project slows down. Developers feel untrusted, and project leads spend all their time chasing up developers to make sure they’re following the process.
  • Situation 3: The customer is unavailable when the developers want him. The developers spot the bottleneck and ask the business to assign more of the customer’s time to them.
  • Desired effect: The customer answers the developers’ questions, and the developers can deliver the right thing.
  • Actual effect: The customer loses touch with the users and with his peers. He doesn’t get to use the end product, so becomes less interested in it. When the product is delivered, the users want many changes and the application no longer solves the needs of the business. The business also suffers because the customer is an skilled expert, and is less available to help the business in its day-to-day work.

These were all real situations which we either encountered or were concerned about happening on our projects. We also brought up some examples outside of the software domain, in the fields of healthcare and education. That prompted a slew of other examples, and the discussion kicked off.

These are the questions we wanted to answer:

  • What stops processes delivering their desired effect? When processes are created that do deliver the right effect, how are they crafted and what is their focus?
  • What meta-practices could we put in place that would help stop perverse incentives from being created?

Human perception skews us towards perverse incentives.

During the discussion, we quickly realised that we have an emotional attachment to weakness. Good practices and implemented solutions are invisible; they don’t impinge on our consciousness. A bad practice or a problem stops us from doing what we want. It demands our attention; demands that we stop what we’re doing and fix the problem. So our focus is skewed, and we automatically allow our efforts to drift. We fix the bad practices, instead of introducing good practices. We confront the problems, instead of finding a route around them.

Some practices which help us avoid perverse incentives.

NB: Whenever I’ve used words like “measure”, I would also like to include curious observation, intelligent reasoning, judgement through experience and blind intuition – anything which is going to affect the changes you make as a result of the measurement. Some of these ways of measuring are more practical than others, depending on the context, and who’s doing it. Pick one that works for you.

  1. Measure reality, not comparative reality.

    Don’t measure how many stories were underestimated; measure how much they were underestimated by. Don’t measure how often the customer is unavailable; measure how many hours he’s there / not there. This will help you to measure the true scope of the problem. It will also help to remind you to…

  2. Keep the balance when measuring.

    If you’re measuring underestimation, also measure overestimation. If you’re measuring how often a customer is unavailable when you want him, also measure how often a customer is available, but unused. If you’re measuring how many developers waste time because they haven’t followed the process, measure how much work gets done because developers don’t have to wait to follow the process.

  3. Consider aspects of the practice you’re about to change that might be beneficial.

    Consider how those meetings in which the customer spends his time might help your project. Measure how much the developers who went off on their own learnt about the domain and the technology, and how much work they got done in the following month as a result. Consider the time that project leads who aren’t chasing up errant developers can spend on other things.

  4. Measure objectives as directly as you can.

    If you want to make your customer happy, get your customer’s feedback, not the devs’. If you want to deliver business value faster, measure how much business value you’re delivering. If you think that rogue developers are slowing down development, look at the change in velocity. Don’t guess or apply reasoning if you can get real figures.

  5. Fit your process to reality, not reality to your process.

    If your process requires your customer to be there and he’s not, maybe change the process – get some BAs, or arrange for another customer to help you. If the estimates are consistently awry, change the process of estimation rather than trying to fix problems that may be unfixable (there is no silver bullet). If your developers waste time doing something that’s already been investigated, make sure the results of investigations are published in a place where they’ll read it.

  6. Concentrate on the things that enable you to get work done.

    Counteract the skew towards weaknesses by deliberately emphasising strengths. We know this works when applied to people and their abilities. It can also work when applied to a process and its practices. Appreciative Enquiry can help create an environment conducive to spontaneity, to getting things done and coming up with imaginative solutions based on things that worked before (and which are therefore less likely to be perverse).

  7. Give feedback, especially if something or someone works well.

    We tend to focus on weaknesses in ourselves. Help the team appreciate their strengths, and avoid focusing on their weaknesses. If there’s something that needs doing and nobody is doing it, don’t berate the team – find someone who does it well and get them to join in.

  8. Provide evidence when providing feedback.

    What evidence is there that the developers have slowed the team down? That the customer’s unavailability has negatively affected the project? How many unfinished or rejected stories were there as a result?

    What did the developers learn as a result of their investigation? What did we learn about our processes? What did the customer provide us with as a result of the meetings?

  9. Make sure that the person in charge of an aspect of the project has an interest in maximising that aspect.

    Don’t praise the IM because his team meets the estimates – praise him because the amount they’re actually doing goes up.

    Make sure the customer assigned to answer the team’s queries has an interest in the final product.

    If you want developers to stop wasting time, introduce them to the customer, and ask the customer to tell them how valuable their work on the end product is – give them the motivation to deliver.

  10. Bring in a fresh perspective.

    Good improvements are often incremental. Sometimes it’s hard to see which practices you’ve put in place that enable a project everyone said couldn’t be done. An external observer can make you re-examine what you do, as well as helping you to pass on those unique practices to other projects and teams.

    Alternatively, try teaching your practices to people who aren’t familiar with your project or methodology. When they ask questions about how that works, given your situation, write the answers down.

This entry was posted in Uncategorized. Bookmark the permalink.

One Response to Perverse Incentives

  1. ext_93136 says:

    Hi Liz!

    Great post.
    While reading it I thought about something a read a while ago in a book, on how to measure people and processes, so I wrote a post about it, if you are interested.

    http://franktrindade.wordpress.com/2008/04/01/tell-me-how-youll-measure-me/

    Cheers,
    Francisco

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s