When it comes to software testing, most IT departments tend to employ a similar strategy: relying on the individual heroics of their testing team. But while sometimes effective, it’s almost impossible to measure. And, without measurement, how can you know its true efficacy?
| Without visibility, the 3 Perils of Software Development—Defects, Delays, and Dollars—can swiftly blow a project off course without warning. Without measurement, departments are often powerless to prevent them. |
To get where you’re going, you’ve got to have a plan. Whether you’re using a map on a hike or plotting your travel itinerary on vacation, that mantra rings pretty true in most walks of life.
Unsurprisingly, the business world is no exception. Every undertaking requires investment; without a plan, there’s no expected outcome to hold it accountable to. If you don’t know how something should be performing, you’ll never really be able to know how you’re doing.
In other words, you can’t manage what you can’t measure. Sounds logical, right?
But here’s the problem: most IT departments have no real way to measure their software testing teams. And without any particular standard to hold them to, they have no real definitive indicator of their team’s performance.
So why aren’t most IT departments up to their eyeballs in production-level defects? That’s simple: they’re getting by on the individual heroics of their software testing team. But while their effort is laudable, they’re not perfect. This is exactly why these same departments will sometimes have trouble meeting their expected deadlines or budget—creating seemingly random aberrations and outliers where the test plan wasn’t met.
But they’re not really that random at all.
Without visibility, the 3 Perils of Software Development—Defects, Delays, and Dollars—can swiftly blow a project off course without warning. And without measurement, departments are often powerless to prevent them. Think of it as regression to the mean: no matter how good your testers are, without a defined process to empower them, some projects will inevitably miss their targets.
But process alone isn’t enough. Just like that map when you’re hiking, without the right metrics, you’ll have no way of knowing whether your software testing team has lost their way. We’re all susceptible to positivity bias; and without a truly objective measurement system, you’re far more likely to overestimate your team’s performance than you are to underestimate it.
So what metrics should you be measuring right now? Well, that really depends on a whole host of factors, largely boils down to one question: how mature is your software testing/QA organization? Just like you wouldn’t value a Ferrari by its average miles-per-gallon, you need to understand your team’s strengths and weaknesses before you can accurately measure them.
Don’t worry—figuring that out isn’t nearly as hard as it sounds. You just need to find a worthwhile third party to do an independent assessment of your organization. Once you have an idea of where you really are, you’ll have a whole lot better idea of the kinds of things you need to be tracking.
Sometimes, the third party can help you out too—for example, every IT Benchmarking Assessment we do includes the various metrics we recommend you track to stay on top of your testing team. But, regardless of how you get there, the important thing is to take that first step.
After all, a plan’s only as good as the destination it leads to. Am I right?
Cheers,
Mike Hodge
Lighthouse Technologies, Inc.
Software Testing | Quality Assurance Consulting | Oracle EBS Consulting
