AgileHolland

Flow to READY, Iterate to DONE

Sat, 04 Jul 2009 17:45 GMT

In a previous blog post I introduced the definition of READY, and I wanted to do another "context" blog post before starting on this one: on the difference between flowing ("kanban") and iterating. However, I had much more to say on the subject than I expected, so the thing kept expanding... I'll gather my thoughts and publish that one later. So for the purpose of this blog post just bear with me: I find that a Product Owner's job is best done in a flow style. And since my dear ex-colleague Lars Vonk told me he was waiting for this post, I'll just explain the how here. Lars, here you go...

Not all backlog items are equal. A backlog item starts out as a rough sketch - usually just the As a.. I want... So That... stanza - and needs to be fleshed out to the extent that it can be picked up by the team in a Sprint. Just like a team has a basic workflow getting stuff to Done, the same applies for the Product Owner role. Scrum does not have any specific support for a Product Owner: somehow the Product Backlog just "happens". In this post I'll try to fill that gap with respect to the process that a Product Owner can follow.

I'll explain a partitioning of the backlog that maps onto a flow, the nature of those partitions and how you proceed through them to get enough stuff Ready for the team to pick up in the next Sprint.

Flow to READY

The overall flow of work through a Scrum team is that the Product Owner role picks up new stuff, gets it READY, and the Team role picks it up to get it to DONE. Note that I explicitly use the word "role" here: team members have a role to play in supporting the Product Owner role to get backlog items to READY.

Partitioning the backlog

A backlog can roughly be partitioned in four areas based on the overall flow:

items that are currently in the Sprint,
items that are Ready,
items that you're preparing to Ready, and
the rest: new stuff.

Of course this is an idealized view of things. In practice the lines are blurred somewhat because the mapping of priority on the workflow steps is not always as clean as you’d like. New items might show up really high in priority, putting it in between the READY items. On the other hand this way of viewing the backlog could be used to enforce the prioritization: something that’s READY could by definition be prioritized higher than something that’s not.

From partitioning to flow

If we take these flow steps and put them side by side, we get the following:

The fact that I've used one color for "New" and "Ready", and another for "Preparing" and "In Sprint" is not a coincidence: "New" and "Ready" are prioritized buffers, "Preparing" and "In Sprint" are Work-In-Progress. Let's go through the flow step-by-step.

Prioritized buffer: New

Backlog items in the New state are the ones you haven't started working on yet, at least not to the extent that you're getting them to READY. Even so, in practice I've seen that it's wise to perform a minimal triage on these items: if you have every mad idea on there, you'll quickly be inundated in an avalanche of items. Stakeholders tend to say "I've got a thousand ideas!", but many of them are just that: ideas. Only a small fraction are actually realistic or useful to implement. This initial triage should be kept simple, but it does put some discipline with stakeholders putting in their requirements. Don't be too worried about stakeholders complaining, in general a stakeholder will appreciate knowing what they need to do to get their requirements in :-). To be put on the backlog as New, a stakeholder should provide the following:

a story solely in terms of the business experience that describes what they experience and what they need without referring to how the product would support it
user story stanza (As a.. I want... So that...)
a (rough) valuation of benefit
a rough indication of size (i.e. cost): small, medium, large. (Note that this is best guesstimated by a team member or the product owner after a chat with the stakeholder: they have more knowledge of the product)

The business urgency gives you a rough prioritization, which enables you to decide which items to pull in first.

Work-In-Progress: Preparing

This step is the big one as far as the Product Owner is concerned: it's where the core Product Owner work gets done.

The Product Owner is the single point of contact for all stakeholders, and this is of course intentional. There needs to be a single focal point for all requirements and prioritization, otherwise it will fall back to the team role and the Product Owner role is all but useless. An unfortunate side effect for our poor Product Owner is that they constantly get bombarded with requirements and pressure from all stakeholders. The result is that a Product Owner is stretched thin trying to deal with it all. I've found that this is where the flow/kanban style really shines: explicitly limiting work in progress is one of the best tools to bringing some sanity in the Product Owner's life.

The Product Owner pulls items into the Preparing step according to capacity. Just like a team pulls in work to capacity and does not change it until the Sprint is over, a Product Owner should pull in a number of items (I've seen two per person in the Product Owner role a lot, don't know yet if that's a general thing, though), work on them until they're Ready, and only pull in new items when a "free slot" opens up in the preparing step.

The Product Owner does not need to (and in most cases can't) provide all information, but is responsible for making sure that someone does, so that the backlog item will get to Ready. This means that a Product Owner will talk with stakeholders to ask them for more information, with the Team to provide an estimation of implementation complexity, and anybody else who is needed to provide clarity and information. This is quite a job, and in larger organizations it's not unusual to see multiple people (often analists) supporting this role.

Because of the explicit step in the flow it is now possible to measure Product Owner performance. The equivalent of velocity in the flow style is cycle time: the average time needed to bring a backlog item from New to Ready. A backlog item that is stuck will be easier to recognize, since it will remain in the Preparing step longer than is usual. It also helps to plan Product Owner capacity. Comparing the cycle time with the a number of backlog items that is consumed per Sprint by the team helps to determine if the Product Owner is going fast enough to keep up.

A Product Owner's speed is not measured in a backlog item's story points but in number of backlog items because the amount of work for each item is roughly the same: every item needs its questions answered regardless of the size. Large backlog items may be more work, but in most cases they will have to be broken up into sizes that are manageable by the team. This translates into more backlog items for (originally) large ones, so large items are factored in this way.

When a backlog item is Ready it can be moved into the Ready buffer.

Prioritized buffer: The Ready Buffer

I have found it very useful to think of the list of Ready items as a buffer. A Product Owner's productivity needs to be such that this buffer is full enough when the next Sprint starts. Tracking the size of the buffer (in story points, since now the capacity of the Team is the relevant one) is a very good way to see if the Product Owner is getting into trouble. You could use a burn-down chart, a burn-up chart, or simply a continuous trend line of buffer size, but I find it a great help that a Product Owner has access to the same type of trend information that is available to Teams when they use burn-down charts.

There are two levels of Ready: each backlog item needs to be Ready, but the backlog needs to be Ready as well, just before the next Sprint. Backlog-Ready means that the Ready buffer is full enough for an iterations' worth of work, and some extra work as "spare change" in case of renegotiation, last minute decisions, insights or priority shifts. In practice going for 1.5 to 2 iterations' worth is a good target. The reverse is also true: if the buffer is really full, more than two Sprints' worth of Ready items, you're likely wasting work since reality will change before you get round to the later items in the buffer (items become "unready"), forcing extra work. In that case you it's better to increase the Team capacity or use the free time for some crystal ball gazing or market research :-).

Work-In-Progress: In Sprint

Of course this step is relatively easy to describe, this is where all the usual Scrum stuff enters the picture :-). As a Product Owner you'll track which items are In Sprint, but your work is not entirely done. In an analogy of a quote on design: "a good design does not depend on one big decision, but on hundreds of little ones", a Team will need you around to unblock them with decisions. During a Sprint a Team will come up with alternatives for details of the implementation. Often these alternatives have an impact on the end result: they might have an easy option but it will not be exactly what was asked, or a team might find a part of the implementation is harder to do than expected. In that case you need to be around to help them forward.

Summary

The backlog can be partitioned in four parts that you can connect with a flow. Since this blog post is getting long enough as it is I'll write another one on how you support this process with a Kanban board and electronic tools.

Bookmark

No Easy Road to Agile Cultural Change

Thu, 02 Jul 2009 10:51 GMT

A number of commentators have written about the challenges involved in migrating an organisation to an Agile culture. Ken Schwaber has estimated that 75% of Scrum implementations will fail to deliver the anticipated benefits. This article looks at some of the reasons why and what can be done to improve success rates. By Shane Hastie

Agile In a Flash

Wed, 01 Jul 2009 11:21 GMT

Many people playfully credit the 3x5 index card as the "agilist's badge". In many ways though this is not an inaccurate or inappropriate; going through a stack of index cards is a often real hallmark of many agile activities. But what about using index cards to learn and remember agile? With their 'Agile In a Flash' project, Tim Ottinger and Jeff Langr want to help people do just that. By Mike Bria

Ultimate Software Named Best to Work For: Scrum Cited as a Reason

Wed, 01 Jul 2009 10:34 GMT

Presentation:Neo4j - The Benefits of Graph Databases

Wed, 01 Jul 2009 10:24 GMT

This presentation covers the definition of a graph database (information structured as mathematical graphs with nodes, relationships and properties) and their advantages when dealing with data that is difficult to fit in static tables, is rapidly evolving, or that has a lot of optional attributes. The flexibility of graph databases better support agile development and schema evolution. By Emil Eifrem

Presentation:Realistic about Risk: Software development with Real Options

Tue, 30 Jun 2009 19:20 GMT

This session introduces Real Options and shows how it can help in running your project. Real Options is a decision-making process based on Financial Option Theory and Applied Psychology that can be used to manage risk. Applying Real Options to software development explains why many of the Agile practices are so successful. By Olav Maassen, Chris Matts

Como fazer o Scrum acontecer em ambientes corporativos

Tue, 30 Jun 2009 07:03 GMT

Como fazer o Scrum acontecer em ambientes corporativos

Esther Derby: Five Reasons to Hire a Coach for Agile Teams

Tue, 30 Jun 2009 02:32 GMT

I run into managers who think that they can coach the team as the team adopts Agile methods. In my experience, this usually doesn't work out very well.

So managers, support the team as they learn Agile methods by hiring a coach! It's a investment in success.

Here are five reasons that coach cannot be you.

1. The team needs someone skilled in XP engineering practices and current on the latest testing tools. If you haven't written code in more than three years, or you've never worked on a team that used all the XP practices you don't have the knowledge to coach the team. Sorry. Doesn't make you a bad person or cancel out your value to the organization. Does mean you are not the right person to coach on XP.

2. You may have a conflict of interest. If you are being measured on delivery dates, its possible that you'll ask the team to cut corners, work extra hours, or drop practices to meet a date. That would be bad. The team needs someone who will hold the process in place and help them hold firm when someone is worried about making a date. That can't be you, if you're the one worried about making the date.

Even if you'd never do that, on some level, the team may not believe that yet, especially if you've ever done so in the past. And if they think you'll cave in the face of a date, they'll cave before you even ask them to. It just works that way.

3. A coach will help the team stick to the process, and guide them away from making adjustments that nibble away at iterative incremental development until they are back in a mini-waterfall.

Agile really does involve a shift in the mental model of software development, and until you have an intuitive grasp of the principles and values, the nibbles will make sense to you, too. You've got to live it for a while before you can coach it.

4. As long as you are there to solve problems and tell people what to do, it will be easy for the team to remain dependent on you. It may feel good to be needed, but in the long run, it doesn't help the team, doesn't help the company, and doesn't help you.

5. You've got a ton of other stuff to do supporting the team, removing obstacles, running organizational interference, and working on the work system. Plus you have to do the budget.

Of course, you don't have to be dependent on outside coaches forever. Once you've established a few Agile teams, you can develop a cadre of coaches made up of your own experienced people.

IBM Rational and InfoQ eBook: Scaling Agile with C/ALM

Mon, 29 Jun 2009 18:11 GMT

IBM Rational and InfoQ preent an eBook, Scaling Agile with C/ALM, "dedicated to all of the functional and dysfunctional organizations that are eager to break down the organizational and cultural silos, and become a finely tuned software delivery machine." The eBook explores the barriers to team integration and scaling and then shows, in detail, how to overcome these obstacles. By Dave West

Jeff Sutherland @ nlscrum

Mon, 29 Jun 2009 16:25 GMT

Last week I co-organized an nlscrum event with a very special guest: Jeff Sutherland. After rushing with him from the airport to our Xebia office, Jeff gave a very inspiring presentation.

Jeff talked about the dramatic difference between those teams that take the red pill (that Morpheus offered Neo in the Matrix), and the blue pill that most people take. After his presentation, many attendees felt like they had just taken the red pill. I hope that this inspiration lasted long enough to have some effect at work, where countless small and big obstacles cause many people's pill to be blue.

After dinner, which was accompanied by a nice Italian ice cream booth, we had a question and answer/discussion session in which Jeff touched many Scrum related topics. Usually at nlscrum events, we host an OpenSpace to give members of the nlscrum community a chance to share experiences. This time, we decided to opt for a different format to give a maximum of attendees the opportunity to learn from and get inspired by Jeff.

The topic that triggered most debate was a discussion about Kanban versus Scrum. According to Jeff, Lean principles and techniques are important to do Agile software development projects successfully. In fact, last month Jeff gave a Deep Lean course with Henrik Kniberg and the Poppendiecks. For those interested, you can find the course material online, including a presentation about Kanban versus Scrum by Henrik Kniberg.

All in all, I'm very glad by the way this special nlscum event turned out to be. We had approximately 65 attendees and got a lot of positive feedback. I hope to see many of them again on our OpenSpaces. So, for all of you in the Netherlands who use Scrum or plan to do so, come to our nlscrum events to get inspired and learn from your peers!

Bookmark

Willem van den Ende: Photo Suggest – photos to go with your blog entry or slide

Thu, 25 Jun 2009 07:59 GMT

We proudly announce Photo Suggest, a web application that helps you find photos with liberal licenses to go with your blog entry or slide Check it out.

Dancing Peacock by Hamed Saber

Here’s why:

As a writer, I want photos to go with my blog entry, so that it looks appealing for readers and inspires me to write more.

Brickpit Ring Walk Bicentenial Park by Louise Docker

As a presenter, I want photos to go with my slide, so the slides have metaphors that make people think, and my presentations look well prepared.

For these two stories we might not have bothered writing a web application – we could use the regular flickr search. However:

As a writer or presenter, I want to easily credit the photographer so that I can fulfill the obligations that go with the license and give viewers the opportunity to see more of their work.

I used to have Super Human Powers by Esparta Palma

We found that finding photos to go with a presentation was easy enough, but collecting the credits and then adding an attribution to the photo in a blog entry or the end of a presentation) often turned out to be a lot of clicks, which meant that we would not add photos to presentations as often as we like…

How does it work?

Photo Suggest queries Flickr and searches for photos with a liberal license (see the about page for a short list) sorted by interestingness. If you click on the link below an image, it takes you to the details page that shows the full credits, license and description together with the image. The reason we called it ’suggest’ is that when you type a keyword, the results are often not what you’d expect, but can more often than not make an interesting contribution to your text.

How did we get here?

With QWAN we try to apply lean and agile principles to everything we do, so we reflect at our ways of working continuously, identify things that add value, and do more of them, as well as things that are wasteful and eliminate them. We started to give more and more presentations to get the word out – we love experiential sessions over anything else, but they do not get us into more traditional conferences. Styles like Presentation Zen led us to do more with images.

Presentations with attractive imagery inspire me more when I do a presentation and they seem to energize the audience as well, so it becomes easier to add experiential elements (small exercises, questions) to a presentation.

Marc Evers and I have been thinking about how to improve our presentations as well as the way we produce them for a while. This led to a bunch of wild ideas, which we used as stories for the new new new! product development game – participants have to plan a ‘presentation 3.0′ project.

With Lasse Koskela I ran a Scrapheap Challenge at XP2009 – participants have to write a working application in half an hour. For that we needed exercises. Some stories from the game seemed well suited, so I did a small experiment – in half an hour I got quite far. Then I called Marc and asked him if he wanted to help me finish it into a working application. We had noticed we were losing some of our “superhuman powers” , so Marc suggested to “start from production”, which meant I talked him through the app while we brought it into version control and production, before adding more features & polish.

From experiment to production took half a day. After a day it was usable enough (but was missing the finish such as about page and stylesheet) for us for blog entries and presentations. When we demoed it during xp2009 a few participants jotted down the url, which encouraged us to polish and publish it.

What Can We Do With Flickr? by Alan Levine

We hope you enjoy it! Your feedback is welcome.

Presentation:Agilists and Architects: Allies not Adversaries Presentation

Thu, 25 Jun 2009 06:39 GMT

As Agile becomes more accepted, concerns from architecture groups are increasing. Traditional ways that architects engage with development groups conflict with Agile methods. This talk describes ways that Agile methods can benefit architects, addresses concerns architects express about agile, and proposes ways that architects and agile development teams can become allies. By Martin Fowler and Rebecca Parsons

Lessons Learned from the UK Agile Coaches Gathering

Thu, 25 Jun 2009 01:00 GMT

Recently, a number of European Agile Coaches gathered in the UK to discuss their craft and share ideas. Attendees included: Rachel Davies, Mike Sutton, David Peterson, Plamen Balkanski, Keith Braithwaite, Duncan Pierce, .... They covered a diverse range of subjects: Effective Coaching Styles, Why Do We Coach? Self Organizing Teams, and others. By Mark Levison

Erwin's Unified Theory Of Everything

Tue, 23 Jun 2009 22:45 GMT

At least software development related. Or "How I think I invented Lean over the weekend"

One of my current customers is a large organization that spends way too much money on software development (as large companies usually do).
To make matters worse I have spend the last couple of week doing code reviews on the software and the code quality is nowhere near what I consider to be acceptable.
So they spend a whole lot of money on bad code.. How am I going to fix that?!
When you are talking to people about increasing quality they immediately remind you of the iron triangle:

Improving Quality means more Money and/or more Time, neither of which is easy to come by in these times. So the only other option is breaking the triangle.

Let's start with the easy one. As the age old saying goes: "Time is money" and this is especially true of software development. Almost all the cost of a software development project is from the people that are working on that project. If we spend less time doing certain things, it will cost less money.
So the first thing we do is replace both Time and Cost by Velocity.

When you then look at the reasons for having high quality software, a lot of them have to do with maintainability and adaptability (see for example ISO 9126). In the complex world we live in today that is not a maintainability in some distant future after we have written and delivered the code. It is not some part of the delivery, but it is the maintainability and adaptability that we need DURING our projects in order not to get bogged down by complexity. It is not something you can easily slap on after the fact, you need to have good quality throughout a project to make that work. Something that Philip Crosby realized in the 1960's with his "Zero Defect" policies.

So Velocity and Quality are not even on the same scale; Velocity is what we want, Quality is a way to improve that, or better said, bad quality is something which slows down Velocity.
That immediately brings forth the image of the "Limits to Growth" pattern as described in Peter Senge's Fifth Principle.

Limits to growth is a pattern in which growth happens, but eventually is limited by one or more limiting factors (usually hidden) that suppress the growth. This phenomenon can be seen on multiple levels, both on an organizational level and project level. I have seen many software development projects that started out great, but eventually ran out of steam and progress slowly went down to an agonizingly slow pace.

So the iron triangle is gone completely. We have only Velocity now, which is the only important thing left. Increasing velocity is only possibly by removing those limiting factors, bad quality being one of them, which of course is exactly what the whole Lean movement is all about. Removing waste (limiting factors) in a process to optimize it. Or in the case of software development, to make it go faster.

Which is exactly why I think Lean and Scrum complement each other perfectly. Scrum is great for measuring the velocity of your sprints and Lean is about improving that velocity.
But the one thing that was still bugging me is that the iron triangle must be right somewhere. It is possible to cut corners and save time and money by sacrificing quality. How many times have we considered telling our project managers it could be done in 15 minutes if we only....
But when you consider that bad quality is just a limiting factor to growth it makes sense. You have succeeded in temporarily speeding things up, but you have 'only' sacrificed some of the velocity of the rest of the project.

I guess it is about time I really read Implementing Lean Software Development: From Concept to Cash by the Poppendiecks to see how much of all this they have figured out way before me

Bookmark

Resource Management in Agile Projects

Tue, 23 Jun 2009 12:43 GMT

Agile projects are known to address the problems of rapid change. These may be changes in market forces, system requirements or implementation technology. One of the change, that does not gel well with Agile projects, is the frequent change of people working on the project. The idea is not to disturb the high performing teams so that they can continue to deliver high velocity. By Vikas Hazrati