Applying Metrics in Agile
In our last blog, we discussed Applying Metrics in Regression Test or Waterfall. Applying metrics in Agile is similar, but different. Almost everybody out there is doing at least some Agile. Whether you’re doing Agile entirely by the book or have a combination of Agile and waterfall, aka “wagile,” that’s ok as the same measuring principles apply. However, there are some key differences you should know when applying metrics in Agile.
Rather than handing the ball from development to testing, developers and testers are on the same team in Agile. These integrated teams of “We” must work together to reach a common goal. Since we are a team of “We”, instead of just looking at testing metrics, we are now looking at sprint metrics (you’ll notice the difference if you compare and contrast these Agile metrics with the Regression/Waterfall metrics we previously discussed). Since Agile is executed in sprints and does not have a full set of requirements, it is harder to scope the overall project. Therefore, measuring and assessing your team’s performance per sprint is crucial as it prevents them from getting too far off course before it’s too late (which is in fact, one of the key points in the Agile Manifesto).
A GIANT benefit of Agile is that the team can learn quickly and frequently. For example, in the first few sprints, you will likely find common bugs as a result of gaps and miscommunications in your process. For example, a developer could have a user story to display a list, but then a tester realizes it must be sorted not just displayed. This immediate feedback will prevent the developer from making the same mistake in future sprints, which in turn, will save you time and money. This continuous feedback loop in each sprint allows your team to quickly learn from their mistakes and adjust accordingly which will improve your overall team’s progress, quality, effectiveness, and efficiency.
Although projects are harder to scope in Agile, there are major benefits of working in two-week sprints. If you apply metrics in Agile, you will learn valuable and applicable lessons at the end of every two weeks. Learning these lessons throughout the process, rather than at the end, allows you to improve as you move from one sprint to the next. If you have a disciplined retrospective (a formal post-sprint meeting), you are not only engaging the whole team by demoing functionality, but by assessing what went well in your process (and giving kudos to the team) and what you should change for next time. How were your metrics? What did you learn?
Hardening Sprint
Here at Lighthouse, we recommend having a hardening sprint after every few regular sprints. Since you only test a few functions in each sprint, a hardening sprint allows you to test the overall functionality up to that point. This also allows you to catch any bugs that were accidentally created while fixing others. If you don’t do a hardening sprint, these bugs might go unnoticed until it’s too late.
At the end of your sprints when you’re feeding into a prerelease regression test, it is the same process as the one we discussed in the previous blog. Although what you will be measuring in Agile is different from regression, the measuring principles are the same. So, what exactly should you be measuring?
Key Agile Metrics
1. Progress Metrics
Think of these as project management metrics per sprint. We planned to get x story points (requirements, bug fixes, tasks, etc.) done, but we actually got y done. Being able to see how you are doing against your plan allows you to learn, adjust, and continue to improve as you move from one sprint to the next. This valuable insight will also enable you to better groom your user stories, better estimate story points, and better plan subsequent sprints, rather than just guessing how much time, money, and effort each sprint will take.
2. Quality Metrics
These metrics indicate the quality of the product. Most teams track some basic quality metrics, but an additional one that we suggest measuring is “Defects Discovered by Sprint.” This shows you how well your sprint team does at removing defects before they are released into production. Another metric that we recommend, “Defect Removal Efficiency by Sprint,” tracks the total percentage of these bugs found over time.
3. Efficiency Metrics
Efficiency measures how quick (or expensive) it is for your sprint team to complete work. In Agile, teams often overestimate how many story points they can complete per sprint. To learn and improve your estimating, we recommend tracking Planned vs Actual story points completed per sprint. Although this is a simple graph, it can have an immediate and significant impact as it holds your team accountable to the original plan. It also shows how much work they are actually capable of completing per sprint which will allow you to create more accurate plans in the future.
4. Effectiveness Metrics
Effectiveness measures how good of a job you are doing. Here at Lighthouse, we use a couple of key effectiveness metrics. One great example is Mean Time to Defect (MTTD) which we discussed in the previous blog.
The lessons you learn from applying metrics in Agile will allow you to become more proactive. Assessing your team’s performance after each sprint, rather than at the end of your entire development and testing processes, enables you to continuously learn and improve. This continuous feedback loop will increase your team’s quality and performance while simultaneously saving you time and money.
To learn more about the benefits of a Metric-Based Test Methodology, please Contact Us and/or read the previous two blogs in this 3-part series. You can also watch our webinar, Metrics That Matter: Keeping Your Software Projects On Course, where we discuss the entire series, as well as demo the Lighthouse Jira-based metrics system.
- Testing Process Maturity – Complete
- Applying Metrics in Regression Test – Complete
- Applying Metrics in Agile – Complete
