Quantify the Impact of Agile App Development

Agile is here to stay. Once the radical alternative to Waterfall development methods, these legacy methodologies are being disrupted and replaced by Agile practices that improve time-to-market, reduce development costs, and produce higher quality software that better meets customer expectations. As the world demands more software, development teams – from scrappy startups to big corporations – are meeting the challenge with Agile.

But while Agile software development projects scale across the enterprise, management is still searching for the best way to gain deeper visibility into these projects. Large organizations cannot rely on the subjective anecdotes of those closest to the work; they require quantitative insight upon which to base business decisions.

Here are some quick tips to quantify the impact of the choices you make during and after your Agile transformation.

1. Start with desired outcomes, not what’s easy to measure

Better measurement leads to better insights, which in turn leads to better decisions and eventually better outcomes. Most people start by measuring what’s easy. But measuring what’s easy can drive the wrong behavior. Let’s take a look at this story about two NBA players.

In 2010, Monta Ellis, with the Golden State Warriors, was the 9th highest scorer in the NBA. Carmelo Anthony, with the Denver Nuggets, was the 8th highest scorer. Measuring individual scoring totals is easy. You would assume that because they were prolific scorers, their teams would win. However, when they were in the game, their teams won less. Scoring is a function of two other measures: 1) the number of shots and 2) the percentage of those shots that go in the basket. It turns out, these two “stars” have low measures for #2, their shooting percentage.

The only reason they are high scorers is because they take more shots. They are literally stealing shots from their teammates who might have a better chance of scoring.

So, while the flow of learning goes from measures to outcomes, the way we think about it should start with outcomes. That’s why we call this ODIM:

better OUTCOMES better DECISIONS better INSIGHTS better MEASURES

The NBA players should focus on the outcome of winning more games rather than being a high scorer. If they used the overall likelihood of the team scoring under various conditions as feedback, it would help them make better game-time decisions to achieve the ultimate outcome of winning. This brings us to our second tip.

2. Think of measurement as feedback, not levers

Frequent feedback is the primary difference between Waterfall and Agile development. Successful Agile projects incorporate short iterations with fast feedback from customers. The key to effective Agile measurement is to think of measurement in terms of feedback, not as the traditional lever to motivate behavior. This often devolves into keeping score, which is where the dark side of measurement starts — avoid it.

There is a subtle, but important, distinction between “feedback” and “lever.” Feedback is something you seek to improve your own performance. Levers are used to influence others. The difference is more in how you use the measure than the measure itself.
For example, healthy use of a burndown chart tells the team if they are on track with their commitment so they can make adjustments in time. The counterexample is a manager using burndown charts to red-flag projects in trouble. While it may drive improvement, nobody wants the red flag thrown at them, so the tendency is to keep the metric in the green regardless of the reality of the situation.

You can’t make better informed decisions if the metrics you are using to gain insight don’t accurately represent reality (see tip #1). Rather, the manager could provide coaching on tools that the team can use to improve their own performance — a subtle but critical difference.

3. A balanced measurement regime or none at all

Balance in Agile measurement (Figure A) includes four cornerstones:

  • Do it fast.
  • Do it right.
  • Do it on time.
  • Keep doing it.


Figure A

Without balance of these four elements, it’s easy to focus on just one. For example, if we focus only on increasing productivity this will likely drive down quality and customer satisfaction.

4. Measure specific outcomes for software

  • Productivity
  • Responsiveness
  • QualityCustomer
  • Satisfaction
  • Predictability
  • Employee Engagement

These six outcomes are the elements of the Software Development Performance Index (SDPI), used to quantify insights about development work and provide feedback on how process and technology decisions impact the development team’s performance. Know what to measure and focus on each individual element.

5. Listen to Experts

Agile has caught the attention of leading industry analysts, asserting itself as a key part of application lifecycle management (ALM) evaluations. These evaluations encompass more than just functionality for developers; they assess commitment to the ALM market, ALM product strategy, corporate strategy, and market presence.

In fact, independent research firm Forrester Research, Inc. recently evaluated the most significant ALM software providers, treating Agile and Lean as critical tests of an ALM vendor’s offering. The report also found that businesses “can no longer accept historical gulfs between business and application development and delivery teams as, increasingly, firms now expect to manage application development and delivery as a business and treat it as a competency.”

6. The Agile perspective for software development metrics

In the move to Agile, overall goals are largely the same as before: to delight users with a quality product delivered in a predictable and efficient manner. Even after your Agile transformation, you will largely do the same “types” of things: analyze, design, code, test, release, maintain, and, yes, measure.

It’s the perspective you take when doing these that is different with Agile.

Larry
Larry started working at Rally Software in 2009 and is currently Rally’s Director of Analytics. He is currently working on his PhD in Software Engineering at Carnegie Mellon.

Prior to starting work full-time on his PhD, Larry served as the Manager of Software Assurance Initiatives for the CyLab at Carnegie Mellon where he promoted the development and widespread adoption of best practices, tools, and methods.

His interests include measurement/analytics, Agile methodologies, software engineering, software craftsmanship, and software assurance. He also have a strong interest in information visualization and has a passion for coding in general which includes programming language technology and design patterns.

Larry Maccherone
Larry started working at Rally Software in 2009 and is currently Rally’s Director of Analytics. He is currently working on his PhD in Software Engineering at Carnegie Mellon.

Prior to starting work full-time on his PhD, Larry served as the Manager of Software Assurance Initiatives for the CyLab at Carnegie Mellon where he promoted the development and widespread adoption of best practices, tools, and methods.

His interests include measurement/analytics, Agile methodologies, software engineering, software craftsmanship, and software assurance. He also have a strong interest in information visualization and has a passion for coding in general which includes programming language technology and design patterns.

The Related Post

Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part Four of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile”   LogiGear Corporation  LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and ...
Build the right test platform including infrastructure, virtual lab and process. Testing embedded software is both similar and dissimilar to application software testing. The first eye-catching thing is that embedded software is significantly less visible to the end user. User interfaces are limited; there may be a console-based text menu, a simple command line interface, ...
Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part Two Continued of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile” LogiGear Corporation LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and ...
One of the features of using Agile methods is the opportunity for continuous improvement within a project. There are a number of improvement opportunities throughout a typical iteration or sprint─over the next few weeks I’m going to walk through a few, starting this week with the Retrospective. Retrospectives are one of the many tools in ...
Team collaboration is essential for testing embedded systems. Developing software for an embedded system often carries more risk than for general purpose computers, so testing is extremely critical. However, there still has to be a good balance between time spent on testing and time spent on development to keep the project on track. As consultants ...
Application development and delivery teams are under constant pressure to release quality features as quickly as possible. CIOs rate delivering applications faster, with higher quality and with strong control on application development as their key priorities. What’s more, supporting this type of agile environment is particularly complex to IT teams that are also tasked with supporting ...
When quality assurance teams and management who have adopted Agile practices first put the ideas to work, they face a significant impediment in unlearning the traditional mind-set and practices that experience in traditional practices has instilled in them. “He who knows to unlearn, learns best.” — Anonymous The following are some of the key aspects ...
Continuous Improvement and Short Feedback loops (think: Test Driven Development; Sprint Demo/Review; …) are at the core of any Agile process. Without a structured improvement process it can be difficult for teams to improve and without improvement we stagnate. For methods like Scrum, XP and et al., Retrospectives are that tool.
There is a multitude of Agile testing techniques that are quite sophisticated. The DAD process can help guide your process of tailoring decisions. Agile developers are said to be quality infected, and disciplined agilists strive to validate their work to the best of their ability. As a result they are finding ways to bring testing and ...
Testing in Agile Part 1 – INTRODUCTION TO AGILE In case you missed the first part of the series in our last magazine issue from Michael Hackett, Agile’s impact on software development teams is huge. For test teams it can be even more pronounced — and good, especially if your existing projects have been problematic.
Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part Two of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile”  LogiGear Corporation LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and offers ...
Maximize the function of your teams The Modern Agile philosophy created by the folks at Industrial Logic is one of the most exciting ideas I’ve encountered in a while. Moving beyond the pre-canned “You must do X to be Agile” mindset that I’ve seen becoming more and more prevalent.

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe