Testing Process Maturity
Have you ever said or thought one of the following:
“Every time we release new features, we break others.”
“We’ve had some major production bugs that have cost us dearly.”
“Our quality is very inconsistent.”
If yes, then your organization could benefit from a metric-based methodology. You might be asking yourself, “why measure software testing?” Well, that’s easy — because you can’t manage what you don’t measure.
However, we know improving your testing process can be easier said than done. So, where is a good place to start? We recommend that you assess your testing and QA processes based upon a maturity model. The following Testing Maturity Levels, defined by the TMMi, categorize the testing processes of organizations into 5 levels. As you move from Level 1 (Ad-hoc) toward Level 5 (Optimized), you will move from being reactive (only focused on defect detection) to being proactive (able to prevent defects from occurring).
Testing Maturity Levels
Level 1: Initial
What is your organization’s testing process? If you are unsure of the exact steps or cannot think of any at all, then your organization is at the Initial level of Testing Maturity. Organizations within Level 1 do not have a written or defined testing process. The testing methods of these organizations tend to be ad-hoc and reactive.
Why does this matter? Without a clearly defined testing process, it is nearly impossible to scope the work, or control the quality of your projects. In addition, how are you expected to repeat successes if you do not know the steps you took to achieve them? Because of this, Level 1 organizations also frequently miss deadlines and exceed budgets.
Don’t let this be you!
Level 2: Managed
Organizations at Level 2 have clearly defined the following aspects of their testing process: Test Policy and Test Strategy, Test Planning, Test Monitoring and Control, Test Design and Execution, and Test Environment. Although the fundamental test policy and strategy should remain the same across all projects, the overall testing process often varies from project to project.
Level 3: Defined
Organizations at Level 3 expand upon Level 2 by requiring all projects throughout the organization to follow the same standards and procedures. In addition to the processes outlined in Level 2, organizations at Level 3 have Test Organization and Test Training. Non-Functional Testing and Peer Reviews are also planned and executed in all projects. Think of Level 3 as being consistent across the entire organization.
Level 4: Measured
Level 4 organizations measure the activities and outcomes at each stage of testing to ensure projects are as defect free as possible. Software Quality Evaluation and Advanced Peer Reviews are also used in addition to the review practices introduced at Level 3.
Level 5: Optimized
At Level 5, organizations assess and optimize all activities to ensure continuous improvement toward defect prevention and optimized quality. As an organization’s testing process matures, their ability to predict and control the quality of their testing improves as well.
Now, you are saying to yourself “OK, I see some benefits of moving from being reactive to proactive, but will I really see tangible benefits?”
The answer is “Absolutely, especially as you add metrics to the process”. The following animated gif shows what happens in an organization as their Testing Process Maturity increases over time.
As you can see, not only does an organization’s ability to predict its quality improve, but the actual quality itself improves as well. As your maturity increases, the plan line lowers, which means you are expecting less defects released into production. Because there are less bugs to find and fix, you will be able to deliver your software systems at a lower cost and a faster time to market. Talk about a win, win, win!
What’s the point of making a plan if you can’t follow it and stay on track? Measuring the various steps of your testing process allows you to better manage them and do just that!
To learn more about how you can improve your organization’s software quality as well as save money and time, keep an eye out for the rest of this 3-part blog series – The Benefits of having a Metric-Based Methodology.
- Testing Process Maturity – Complete
- Applying Metrics in Regression Test (or Waterfall)
- Applying Metrics in Agile
Or, click the link below to attend the webinar we are hosting on Wednesday, July 20 at Noon (EDT) where we will be discussing all 3 parts of this series. We hope to see you there!
