Solve problems with experiments

Lately I’ve been exploring tools that can help teams more successfully solve problems. This article is about how you can use experiments to help you solve problems more effectively.

Your convictions are more likely assumptions
In product development we are finally starting to acknowledge that the convictions we have about our product actually are assumptions (that often also turn out to be wrong). The realisation of this is one of the reasons to why we adopt lean startup or create impact maps. We want to find out if our convictions are wrong early because we want to discover what our users or customers actually want faster i.e. profit faster. That’s why we run experiments when developing a product.

Experiments help teams understand what the problem is
When it comes to problem solving I see teams jumping to conclusions and not spending enough time exploring what the problem is.

I consider using experiments when teams:

• fail to improve despite being engaged in retrospectives
• fail on solving a problem over and over
• fail on understanding what the problem actually is
• successfully solves the wrong problem
• bring up the same problems in 3 or more succeeding retrospectives
• agree on what problem to solve and which actions to take but repeatedly fails
• try to solve all problems at the same time
• try to find the perfect solution to their problem and get stuck in analysis paralysis
• work in a simple or possibly complicated environment

However I do not limit myself to those situations, they just trigger me to consider using experiments as a part of a teams problem solving process.

One of my teams recent experiments
As a daily stand-up was nearing its end Carl, a developer in a team I coached, complained about backlog, which they called the later-log, having no visibility.

“It’s crowded with tasks and we’ve actually completed some of them. Some tasks are more than 6 months old and some are a few days old. And we mostly write new tasks for everything so why do we even spend time on adding stuff in our backlog?“ The whole team was in agreement.

Help the team look ahead when they’re agreed on the problem
“How would you like it to be?” I asked Carl and the team. After a few minutes of intensive discussion the team said “We want our later-log to only contain work we will do and nothing that’s older than at most a few months because if it’s older we probably won’t do it. We also want to take on more work from the later-log because we do think the work there is important.”

Generate solutions when the team agrees on the problem and the goal

I asked the team to think of a few different ways they could solve this problem and they came up with a many suggestions; weekly grooming sessions, daily cleanup, FIFO, and some other ideas. The team started discussing which of these ideas would be the best and I noticed that they were unable to reach a decision so I proposed that they run an experiment.

“How about trying out One of the ideas you’ve come up with today for a month to see if it improves the situation for you? If you don’t like it you can always go back or try something else.”

And here’s where something happened to the team. They were suddenly ok with all the different ideas even though the ideas might not be perfect. They quickly decided to try something related to FIFO and after a few minutes they had decided on a new process to try.

The team now knew what problem they wanted to solve, what they were aiming for, and how they were going to try to solve the problem.

Help the team understand what success looks like
“Let’s say this experiment is successful, how would you know it was?” I asked. “Well, if we complete more work from the later-log and it only contains work that’s 5 weeks or newer we’d say it was a success”.

We then discussed what long term effects we hoped to see from this, what we hoped would not happen, and when we would follow up on this experiment. All in all it took the team 30 minutes to agree on the problem and to decide on how to solve this.

I compiled my notes from the teams discussion and put it up on their board.

Experiment – Complete more work from the backlog
Background: We have many tasks in our later-log (product backlog) that are 3 months or older that are important but never gets done. We consider the time we invest in writing and discussing tasks older than 5 weeks to be wasteful. We think the reason that old tickets do not get prioritized is because we do not go through the backlog regularly and we forget the tickets. Our now-log (sprint backlog) is filled by the team every monday and mostly we do not take any tasks from the later-log.
Goal: That our later-log always has relevant work and we complete work from it.
What we will try:To complete more work we will divide the later-log into 5 states:

1. Kill the ticket or do it this week
2. 3-4 weeks old work
3. 2-3 weeks old work
4. 1-2 weeks old work
5. 0-1 week old work

Every Monday during planning we will move tasks one step up. When a task is in “state 1” we will either move it to this weeks backlog (now-log) or kill it.
Success criteria: This experiment is successful if there are no tasks in the later-log that are older than 5 weeks, and if we take on tasks from the later-log every week.
Desirable long term effects:

• less time spent writing tasks that we will not complete
• actively decide what to do and what not to do (i.e. not forget important tasks because they drown in a cemetery of stickies)
• that it’s easy to get an overview of what the work in our backlog is

Undesirable long term effects:

• We only complete work from the later-log even though there might be more important work to do
  We kill everything on the later-log just to follow the process

This experiment runs between 2014-09-15 and 2014-10-20.

Experiments have many hidden values
There are several reasons for why I like experiments, here are a few:

• The structure makes sure that everyone is on the same page – you don’t want some to  discuss success metrics when half the team hasn’t even understood the problem
• They align the team, suddenly everyone knows what problem is and what to try
• There’s a clear goal owned by the team
• The team doesn’t have to get it right straight away – speed of iteration is more important than quality of iteration.
• They allow teams to followup on if they solved their problem
• Undesirable effects help the team recognize if they’re going down the wrong path early which gives the ability to revert decisions that worsen things

Wait, what’s the difference between A3 thinking and experiments?
Both A3 thinking and the experiment format described here are based on PDCA. My short answer is that the deciding factor is the complexity of the problem. If you are dealing with simple* problems e.g. within team collaboration, group dynamics, or a teams process A3 thinking is going to be unnecessarily heavy and slow.

*simple is not the same as easy

Are experiments anything for you?
Now think about your team. Do you feel or think, if you’re a thinker, that you and your team solve problems effectively or do you find yourself discussing the same problems over and over without ever solving them?

Experiments might be valuable for you if you are dissatisfied with you and your teams’ ability to solve problems. Here’s an experiment template that you can use should you ever want to run an experiment for yourself whether that’s for learning or for implementation, with your team or your organization.

If you do run experiments, please let me know. I would love to hear how it’s working for you.