AgileHolland

Scrum for maintenance? Old habits die hard.

Sun, 21 Mar 2010 15:15 GMT

It is difficult choosing Scrum as a project delivery method. It's also difficult implementing Scrum in an organization that is not used to an agile way of thinking. One might think though, that after a year working with Scrum and successfully delivering a new system, Scrum should be at least considered as a viable option for maintenance [...]

Presentation ‘Agile Systeemontwikkeling Met Scrum' (Dutch)

Sun, 21 Mar 2010 15:15 GMT

Last Thursday I gave a presentation about ‘Agile system development with Scrum' to an audience of project managers, business consultants and more technically involved IT people. My main purpose was to provoke a ‘paradigm change' since most people in the audience aren't involved in agile system development currently. So a large part of my presentation was about predictability [...]

Whitebook: PRINCE2 combined with Scrum

Sun, 21 Mar 2010 15:15 GMT

Last year I published a Whitebook in Dutch about the combination of PRINCE2 and Scrum. I also promised an English translation. It is long overdue, sorry about that, but finally here it is. Why combining PRINCE2 with Scrum? I think Scrum is great and we all should use the Scrum approach whenever possible. The ‘whenever possible' bit [...]

Presentation knowledge session PRINCE2 with Scrum (Dutch) published

Sun, 21 Mar 2010 15:15 GMT

My (Dutch) presentation on the combination of PRINCE2 and Scrum is published on slideshare. One warning though. In 1.5 hours it isn't possible to explain all the details of both PRINCE2 and Scrum, so I didn't. I told a little bit about Scrum and almost nothing about PRINCE2. The presentation was mostly about the specifics of [...]

Where Gemba is found in projects

Sun, 21 Mar 2010 15:15 GMT

Three words: Gemba, Kaikaku, Kaizen. Gemba means ‘the place truth can be found'. Kaikaku means ‘radical improvement'. Kaizen means ‘gradual improvement'. Projects are all about change. Usually we're talking about a large change (Kaikaku). That's why it's a project: a temporary organization with a specific goal. You might also argue that an agile project approach also has some Kaizen characteristics: [...]

Published Whitebook ‘PRINCE2 + Scrum: 1 + 1 = 3′ (Dutch)

Sun, 21 Mar 2010 15:15 GMT

Yesterday I published my latest Whitebook ‘PRINCE2 + Scrum: 1 + 1 = 3‘ (in Dutch) in which I make the case that PRINCE2 adds value to Scrum. This Whitebook is largely based on my presentation ‘Agile results with PRINCE2 control', which in turn is based on my latest real life project experiences. I'll try to [...]

Presenting at Dutch seminar ‘Agile project results with PRINCE2 Control' (Nov 26th)

Sun, 21 Mar 2010 15:15 GMT

I'm happy to announce that I'll be presenting (in Dutch) at seminar ‘Agile project results with PRINCE2 Control' Thursday the 26th of November. I'll share my experiences combining PRINCE2 with Scrum and explain what makes this combination very successful. Also, we have reserved plenty of time for questions and discussions, so hopefully there is a lot [...]

Project Boards are agile

Sun, 21 Mar 2010 15:15 GMT

Scrum Masters normally do not have access to a Project Board but in my opinion, they should. With PRINCE2 the Project Board is accountable for the project's success. Within the Project Board there are three separate roles: The Executive is accountable for the project's Business Case. The Senior User represents the people whom will use the [...]

Where Scrum sucks

Sun, 21 Mar 2010 15:15 GMT

I do love Scrum. But at the same time Scrums sucks at a lot of areas from a business point of view. In my opinion the Scrum process is great to get things done. It's great to get a motivated team with focus, it maximizes creativity and delivers value. The business side of Scrum however is almost [...]

Combining PRINCE2 with Scrum is officially ‘allowed'

Sun, 21 Mar 2010 15:15 GMT

The combination of PRINCE2 and Scrum has received the stamp of approval by the auditor.

Velocity planning or capacity planning (or both)?

Sun, 21 Mar 2010 15:15 GMT

Traditionally (if that even exists with agile project management) the team's velocity is expressed in the number of story points delivered per iteration (sprint). In an ideal (theoretical) world, the next iteration's velocity should be the same (or slightly better) then the last one. In the real world this is not the case. One cause is the [...]

Lean Business Process Design

Sun, 21 Mar 2010 15:15 GMT

In most projects some amount of analysis is required. With an agile method like Scrum there's no room for a separate analysis phase. It is assumed that analysis can be done concurrently with design and build activities. This is often true when the business process is known. Sometimes also the business process needs to be determined. This is [...]

Implementing Scrum, ‘crash' start or controlled start?

Sun, 21 Mar 2010 15:15 GMT

Most people advocate a “crash” start when implementing Scrum. Don't think, just go! It's very difficult to explain the advantages of Scrum when the customer is not used to “agile” thinking. Because Scrum delivers tangible results in short increments one could think it's possibly better to let the results do the talking. Here in the Netherlands the [...]

Psychology of a timebox approach

Sun, 21 Mar 2010 15:15 GMT

Timeboxes are widely used in agile and non-agile projects. The idea is that scope is (or should be) limited to the amount that is feasible within the given timeframe. The assumption is that the team is capable of limiting itself to tasks that can be completed within the timebox. But what happens when the teams discovers that [...]

Managing expectations with Scrum

Sun, 21 Mar 2010 15:15 GMT

One might say that managing expectations becomes much easier with an agile approach like Scrum. In stead of waiting for months before there is any result visible the Scrum approach delivers results each iteration (each sprint), thus building trust and creating positive energy. I would argue that managing expectations is much more challenging than one would think [...]

Do Scrum projects need a business case?

Sun, 21 Mar 2010 15:15 GMT

The Scrum method does not explicitly state that a business case is needed. So why should you have one? First of all, let me ask you another question. “Do Scrum projects have a business case?” Well of course they do. Every project has a business case. A “reason why”, a “justification of”, a specific goal. Sometimes a business [...]

Skiing as an agile vs waterfall metaphor

Sun, 21 Mar 2010 15:13 GMT

I was asked by one of my clients to give a short introduction into Agile. As we did not have an appropriate presentation for this kind of audience and knowledge level I decided to create a new presentation. And while I was thinking of a good metaphor to compare traditional waterfall against agile methodologies the [...]

Slow change in agile consultancy

Sun, 21 Mar 2010 15:13 GMT

We as Agilists are extremely result driven: delivering value to the customer as soon as possible is the axle around which our work and vision revolve. This can help us but also hamper us in the process of bringing Agile to a non-Agile environment. Being aware of this may already help us be more effective [...]

Automatic testing Oracle Service Bus using Hudson, maven and SoapUI

Sun, 21 Mar 2010 11:01 GMT

A lot of current projects are implementing some sort of service based architecture. Testing in this architecture becomes more complex. When implementing an OSB project with Scrum you test-automation is imperative. Scrum will require more frequent testing of your system. This is only feasible (in time and money) when you automate as much as possible. Using [...]

An Alternative to Certifications

Thu, 18 Mar 2010 20:59 GMT

The Agile Skills project is a resource for establishing a baseline of skills that an Agile Developer needs. It provides an evolving repository and place to start learning about these skills. By Mark Levison

Organizational Barriers and Impediments to Big Scrum Implementations

Thu, 18 Mar 2010 08:18 GMT

Orlando Scrum Gathering blog/link below

Thu, 18 Mar 2010 08:11 GMT

Community of Scrum Gathering, Orlando 2010 Reference list of Open Space Sessions

Before You Make the Leap to Agile - Ten Weaknesses of the Agile Methodology

Tue, 16 Mar 2010 12:00 GMT

It's been nearly a decade since Martin Fowler, Ken Schwaber and fifteen other experts in the software industry wrote the Agile Manifesto outlining an approach to software development radically different from the Waterfall model that dominated the 1980's and 1990's. Since that eventful time in 2001, Agile software development methodologies, including the use of Scrum, XP (Extreme Programming) and Crystal, are all the rage throughout business and government, attracting praise like a miracle drug. As of early 2010, Agile is till one of the more popular buzzwords in software development.

If you're in the captain's chair as a C-Level IT executive (CIO/CTO), the head of software development, the head of your Project Management Office (PMO) for your business, or the lead project manager in a small organization, you may be considering the leap to Agile seriously. The good news is that Agile is working well for many businesses and on many software development projects.

The bad news is that Agile is not the magic bullet many are claiming it to be. As many successes as there are in Agile development, there are also many failures in both the adoption and use of Agile in software development. (Here's a list here of some cautionary tales, provided to support my statements, not to complete turn you away from Agile: The Great Pyramid of Agile , Cargo Cult Methodology: How Agile Can Go Terribly, Terribly Wrong, Agile: Anatomy of a Failed Project , A Scrum Project That Failed.)

The fact is that seasoned business and IT leaders, particularly those at larger businesses, should take a closer look before plunging in to the Agile world with reckless abandon. Agile has a number of weaknesses that many disciples of Agile fail to acknowledge. After all, they're in the business of developing with and helping you to adopt Agile, so they're not likely to tell you about its problems and limitations.

Together with my friends and fellow associates at Cedar Point Consulting, we've come up with this list of Agile weaknesses, which apply to Agile as it is most commonly implemented:

True Agile is rarely practiced. Long-time practitioners will tell you, “It's Agile – not agile”. Without a doubt, the biggest single problem I have encountered in Agile-practicing shops in the past decade has been that too few people truly understand and practice Agile. In many cases, software developers, development managers and consultants alike mistake Agile for its lowercase sibling, agile, and assume that Agile is all about flexibility and absence of process.This is far from the truth. Agile has formal rules and structure, though they are quite a bit different from those of other development approaches. Agile is iterative, it is adaptive and it is supported by some outstanding tools and techniques, like burn-down charts, product backlogs, user stories and stand-ups. Most important, Agile is not anarchy. It does not mean that everyone does whatever they want and there's no sense of organization, despite the fact that you may feel this is the case.
- Some appropriate descriptions of Agile can be found at these sites, where you can better understand Agile, but you can also see that the Agile has quite a bit of history, rules and structure.
- A Gentle Introduction to Agile: http://www.agile-process.org/
- Wikipedia's entry on Agile: http://en.wikipedia.org/wiki/Agile_software_development
- The Twelve Principle of Agile: http://www.agilemanifesto.org/principles.html
- Agile Defined by Scott Ambler: http://www.agilemodeling.com/essays/agileSoftwareDevelopment.htm
Heavy customer interaction is essential. Reflected in principles four and five of the Agile Manifesto, heavy customer interaction is one of the biggest benefits of Agile, but it also becomes a weakness in some environments. Consider, for example, situations where the customers or users of the software being developed do not have the time available to meet with members of the software development team on a frequent basis? What if the key customer is the CEO, who often has more important matters to address? What if you're not permitted to talk with the users?One important thing to know about Agile development teams is that they are high maintenance – they work quickly, but they also require much more time and attention from the business side to be able to work quickly. If that time cannot or will not be spared for their benefit, using an Agile approach will bring little gain over a Waterfall approach.
Agile thrives with co-located teams. In a typical Agile environment, the Agile development team and the business users are located in the same physical location, such as the same floor and cube space, to increase interaction within the team and with the customer. This technique is highly effective, but it is also not always practical. In many cases, the line-of-business does not have the space to temporarily house the development team members. In other cases, the development team is national or international, so it is simply impossible or cost-prohibitive to have some members of the team on site.
Agile has difficulty scaling for large projects and large organizations. When run properly, an Agile development team is typically in the three-to-eight person range, being made up of the software product manager, a team leader (often the Scrum Master), one-to-three developers, and in some cases a designer, an business analyst and a tester. In many cases, the designer, analyst and test roles are shared by the leader and the developers, so the team could be as small as three people. With its heavy emphasis on customer interaction, self-organizing teams, verbal-communication over-written-documentation, prototyping and requirements flexibility, it becomes extremely difficult to coordinate work as an Agile team grows. For example, one developer may be working on an order entry screen that relies upon data in a product catalog or inventory management system being created by an entirely different group within the team. These two need to talk to make certain their two parts of the system work well together, or they'll end up with two incompatible parts, which is not a problem with a small development team sitting side by side. Assume, instead that the two developers share work with thirty other team members and they are thousands of miles apart. How likely are they to discuss the shared work area? How hard will it be to uncover their shared interest via daily standup meetings? The scalability problem with Agile is not minor and it shouldn't be overlooked. I've seen this problem wreak havoc in three businesses and three large-scale projects to-date, so it's not to be taken lightly.
Agile is weak on architectural planning. While software architecture is not entirely like building/construction architecture, they do share some qualities, including the need for a well-defined architecture up-front. In construction, a good building architect determines the structure of the building, the materials to use and integrates the architecture into the landscape long before the first nail is hammered. Similarly, a reliable software architecture and the software platform are selected before the heart of the software is developed. Otherwise, there's a good chance much of the work will need to be thrown away, much like a make-shift shack in a third-world shanty town. The XP approach does have architectural spikes, but on large scale projects it may take many weeks or months to choose and verify that a particular set of technologies is the most appropriate one, something that can hardly be done in a few days within a single iteration. Yet, the decision on which technology platform to use and how to structure the system is so critical, because it not only affects the success of the overall project or program, it binds the business to that architecture for years to come.This weak-architecture problem is only an issue when the architecture and the platform are not known or pre-determined. In many larger businesses, software architecture and platform selection are done separately from software development via enterprise architecture and architectural roadmaps, which often takes the burden away from the Agile software team. However, if the platform is not known prior to the start of the project and the architectural approach is new or unproven, then the Agile software approach will struggle to define and deal with architecture.
Agile has limited project planning, estimating and tracking. There are tools available in the Agile arsenal for planning estimating and tracking, but as Agile proponents will tell you, there's very little emphasis on these core aspects of project management. By design, Agile minimizes planning through the use of backlogs – prioritized to-do lists of software product features – and by fixing deadlines instead of scope. In most cases, estimating level-of-effort is done for each iteration (sprint), with only rough estimates for for each release.Efforts to track progress vary – some projects ask the developer to mark a specific task on the backlog as “done”, while others ask team members to report hours while tracking which work items are designed, developed an tested as though they flow through a miniature life-cycle in each iteration.In most cases, this lightweight planning, estimating and tracking process suits small, non-strategic projects well. But often is not acceptable when the customer is a third-party who requires scope and cost defined via contract or a senior manager who wants to know what will be provided, when it will be done and how much it will cost.
Agile requires more re-work. Because of the lack of long-term planning and the lightweight approach to architecture, re-work is often required on Agile projects when the various components of the software are combined and forced to interact. In Agile-speak, this is roughly called “refactoring” and it is both expected and budgeted for in each iteration after the first. Certainly, refactoring has its benefits in that one team member does not have to wait for the other to begin work. However, in a worst-case scenario, major portions of code may need to be re-written when two or more developers are not in sync, resulting in higher and higher re-work costs as the number of iterations increases.
Challenges making contractual commitments. For many projects, the client and/or senior management want commitments about the triple-constraints – what will be provided (scope), when it will be provided (time), and how much it will cost (cost). For Agile projects, in particular, it is extremely difficult to prepare estimates for fixed-bid contracts and its not uncommon to see senior managers pound their fists on the table when they fail to received detailed estimates.
Agile increases potential threats to business continuity and knowledge transfer. By nature, Agile projects are extremely light on documentation because the team focuses on verbal communication with the customer rather than on documents or manuals. As a result, a switch of a single team member, much less an entire development team, away from one product and over to another can significantly impact the organization's ability to maintain and improve that product. Much of the knowledge resides in the team's head and is gone when they are.
Agile lacks the attention to outside integration. Both large software development projects and small projects in large environments require multiple points of integration with other systems in their universe, such as sharing data with a downstream system (from order processing to accounting) or retrieving data from another system (such as inventory quantities to order processing).

In most cases, these integration points need to be identified early, then they need to be clearly defined so that both sides can develop the two software components to work well together. The problem is worsened with third-party integration, when another business needs to integrate with you or you need to integrate with them. In these cases, contracts and service level agreements come in to play, requiring lawyers, approvals and frequent meetings before the first line of code is written, pushing the total completion time up by 2 to 10 times.

Because Agile teams often do not invest the time in identifying and designing the integration points with other systems in advance, the need for an integration point can become a last-minute surprise that often requires re-work, additional time, removal from scope, or a poor-quality product. For many of us, none of these options are acceptable, which explains the weakness of Agile in this area.

With all the strengths of Agile methodologies, these ten weaknesses can (and should) prevent some businesses from adopting Agile in its most common form. While Agile is very popular with consultants, IT journals and on the Internet, it's not well suited for every business and every project (see “Agile Home Ground” versus “Plan-driven Home Ground”).

To summarize, this methodology positioning map should give you a rough picture of where Agile thrives compared with Waterfall and RUP:

If, with the list of weaknesses in mind, you're still thinking about making the move to Agile or you're using Agile already, there are quite a few ways to offset Agile weaknesses. In my next article, I'll describe some key improvements being made to Agile as it matures to handle larger, more complex software development projects and adapts to go head-to-head with the more traditional methodologies.

Donald Patti is a Principal Consultant with Cedar Point Consulting, a management consulting practice based in the Washington, DC area, where he advises businesses in project management, process improvement, and small business strategy. Cedar Point Consulting can be found at http://www.cedarpointconsulting.com.

Johanna Rothman: Wage Cost and Project Labor Cost

Tue, 16 Mar 2010 09:29 GMT

I’ve been working with teams who want to move to agile. Some people on their teams are in another location, where the salaries are cheaper.

It’s difficult to get agile started with a geographically distributed team. If everyone’s distributed, it’s easier than if just some people–especially if they are all one function, such as developers or testers–are. Or, if the people on the team know what it’s like to work in a highly collaborative environment, it’s ok, but not as good as when everyone is all together in one location.

The problem is that many managers have confused wage cost with project labor cost. Wage cost is a part of run rate, what it costs to keep the project alive for a week at at time. Yes, cheaper salaries will reduce the project run rate.

The problem is: what happens if the geographically distributed project takes longer to deliver the project? My experience says that all the geographically distributed projects I’ve met take longer to complete. The lack of being all in one place made a particular team take longer to deliver running, tested features. Here’s an annotated value stream map that represents this organization’s delays:

Wage cost is certainly lower in some parts of the world. But the only measure of productivity is running, tested features. If your project team takes longer to complete features, then you have a larger project cost.

Before everyone gets so excited about bits and pieces of remote team members, ask yourself, “Are we building in delays that will cause us to take longer to complete running, tested features? What will those delays cost us?” Now you can start to look at wage and project cost and make decisions that will make sense for your team–whether that means moving to agile or not.

Tweet This Post

Dale Emery: Four Layers in Automated Tests

Mon, 15 Mar 2010 19:37 GMT

I’ve known for a while that when I automate tests, layers emerge in the automation. Each chunk of automation code relies on lower-level chunks. In Robot Framework, for example, tests invoke “keywords” that themselves invoke lower-level keywords.

The layering per se wasn’t a surprise, because automated tests are software, and software tends to organize into layers. But lately I’ve noticed a pattern. The layers in my automated tests center around four themes:

Test intentions
System responsibilities
Essential system interface
System implementation interface

Test intentions. Test names and suite names are the top layer in my automation. If I’ve named each test and suite well, the names express my test intentions. Reading through the test names, and seeing how they’re organized into suites, will give useful information about what I tested and why.

For example, in my article on “Writing Maintainable Automated Acceptance Tests” (PDF), I was writing tests for a system’s account creation feature, and specifically for the account creation’s responsibility to validate passwords. I ended up with these test names (see Listing 7):

Rejects passwords that omit required character types
Rejects passwords with bad lengths
Accepts minimum and maximum length passwords

In an excellent video followup to my article, Bob Martin organized his tests differently, using FitNesse. He grouped tests into two well-named suites, “Valid passwords” and “Invalid passwords.” Each suite includes a number of relevant example passwords, each described with a comment that expresses what makes the example interesting.

Every test tool that I’ve used offers at least one excellent way to express the intentions of each test. However expressed, those intentions become the top layer of my automated tests.

System responsibilities. A core reason for testing is to learn whether the system meets its responsibilities. As I refine my automation, refactoring it to express my intentions with precision, I end up naming specific system responsibilities directly.

In my article, I’m testing a specific pair of responsibilities: The account creation command must accept valid passwords and reject invalid ones. As I refactored the duplication out of my initial awkward tests, these responsibilities emerged clearly, expressed in the names of two new keywords: Accepts Password and Rejects Password. Listing 7 shows how my top-level tests build on these two keywords.

Essential system interface. By system interface, I mean the set of messages that the system sends and receives, whether initiated by users (e.g. commands sent to the system) or by the system (e.g. notifications sent to users).

By essential I mean independent of the technology used to implement the system. For example, the account creation feature must offer some way for a user to command the system to create an account, and it must include some way for the system to notify the user of the result of the command. This is true regardless of whether the system is implemented as a command line app, a web app, or a GUI app.

As I write and refine automated tests, I end up naming each of these essential messages somewhere in my code. In my article, Listing 2 defines two keywords. “Create Account” clearly identifies one message in the essential system interface. Though the other keyword, “Status Should Be,” is slightly less clear, it still suggests that the system emits a status in response to a command to create an account. (Perhaps there’s a better name that I haven’t thought of yet.) Listing 4 shows how the higher-level system responsibility keywords build upon these essential system interface keywords.

System implementation interface. The bottom layer (from the point of view of automating tests) is the system implementation interface. This is the interface through which our tests interact most directly with the system. Sometimes this interaction is direct, e.g. when Java code in our low-level test fixtures invoke Java methods in the system under test. Other times the interaction is indirect, through an intermediary tool, e.g. when we use Selenium to interact with a web app or FEST-Swing to interact with a Java Swing app.

In my article, I tested two different implementations of the account creation feature. The first was a command line application, which the tests invoked through the “Run” keyword, an intermediary built into Robot Framework. Listing 2 shows how the Create Account keyword builds on top of the Run keyword (though you’ll have to parse through the syntax junk to find it).

The second implementation was a web app, which the tests invoked through Robot Framework’s Selenium Library, an intermediary which itself interacts through Selenium, yet another intermediary. Listing 8 shows how the revised Create Account keyword builds on various keywords in the Selenium Library.

Translating Between Layers

Each chunk of test automation code translates an idea from one layer to the next lower layer. Listing 7 shows test ideas invoking system responsibilities. Listing 4 shows responsibilities invoking messages in the essential system interface. Listings 2 and 8 show how the essential system interface invokes two different system implementation interfaces.

Each of the acceptance test tools I use allows you to build layers like this. In FitNesse, top-level tests expressed in test tables may invoke “scenarios,” which are themselves written in FitNesse tables. And scenarios may invoke lower-level scenarios. In Cucumber, top-level “scenarios” invoke “test steps,” which may themselves invoke lower-level test steps. In Twist, “test scenarios” invoke lower-level “concepts” and “contexts.” Each tool offers ways to build higher layers on top of lower layers, which build upon yet lower layers, until we reach the layer that interacts directly with the system we’re testing.

In the examples in my article, I chose to write all of my code in Robot Framework’s keyword-based test language. I defined each keyword entirely in terms of lower-level keywords. I could have chosen otherwise. At any layer, I could have translated from the keyword-based language to a more general purpose programming language such as Java, Ruby, or Python. The other tools I use offer a similar choice.

But I, like many users, find these tools’ test languages easier for non-technical people to understand, and sufficiently flexible to allow users to write tests in a variety of ways. In general, I want as many of these layers as possible to be meaningful not just to technical people, but to anyone who has knowledge of the application domain. So I like to stay with the tool’s test language for all of these layers, switching to a general purpose programming language only at the lowest layer, and then only when the system’s implementation interface forces me to.

A Lens, Not a Straightjacket

When I write automated tests for more complex applications, there are often more layers than these. Yet these four jump out at me, perhaps because each represents a layer of meaning that I always care about. Every automated test suite involves test ideas, system responsibilities, the essential system interface, and the system’s implementation interface. Though other layers arise, I haven’t yet identified additional layers that are so universally meaningful to me.

These layers were a discovery for me. They offer an interesting way to look at my test code to see whether I’ve expressed important ideas directly and clearly. I don’t see them as a standard to apply, or a procrustean template to wedge my tests into. They are a useful lens.