The Glitch(es) That Stole Christmas

Imagine trying to launch a new website for the holiday season, and your offshore developer delivers defect-laden code with far more bugs than you expected. Do you launch the site anyways—risking widespread user frustration and revenue loss—or do you correct the defects while missing the busiest shopping season of the year?

The Glitch(es) That Stole Christmas

Photo courtesy of Parée (Creative Commons – Modified)

Thanks to buggy code from their offshore developer, our client faced a conundrum: make the deadline and release the buggy site to a cacophony of bad press, or release after the holidays and sacrifice potential revenue to give users the quality they expected?

Fortunately, they waited to get it rightand saw their online sales increase by $150m. 

For most of us, the start of Christmas season is one of the most wonderful times of the year. Whether you’re spending precious family time, connecting with long-lost friends, or just shopping until you drop, few people would describe the season as anything but one of the happiest, most relaxing times of the year.

But for one of our past clients, this picturesque description was nothing more than mere fantasy.  Thanks to buggier-than-expected code from their offshore developer, their new website was in serious danger of missing its holiday launch.

It all started with a panicked call from a frustrated international retailer. At the time, the eCommerce market was exploding—and their current site just wasn’t cutting it anymore. But while they’d spent considerable time designing a site with a slick, user-friendly interface and scoped the project well enough to give themselves plenty of time to succeed, they failed to account for one thing: the quality of their offshore developer.

Sure enough, as soon as the code was delivered the retailer knew they had a problem on their hands. The defect-laden code was far buggier than they’d expected, and they needed a team of professional software testers to assess the damage and get the project back on track. We got to work immediately—running 13 test cycles against 20 builds over the course of a lengthy testing period which saw our testers working double shifts to meet the client’s schedule while executing 10,436 test cases in total. 

Our results were prolific, and almost certainly not what our client was hoping for.

Since the developer claimed they were a CMMI Level-5 organization, we thought we were being conservative when we used our proprietary defect prediction drivers to estimate the defects that would’ve been introduced by a CMMI Level-3 organization. Unfortunately, that estimate proved far too optimistic, as we found over five times more than that—discovering 939 unique defects in all, 293 of which were critical.

With the defect-related rework mounting, our client faced a conundrum: release the buggy site to a cacophony of bad press (and user frustration), or release after the holidays and sacrifice potential revenue to give users the quality they expected?  In the end, they made the  customer-centric choice to delay—unveiling their site to a $150 million boost in online revenue, and in plenty of time for next year’s holiday. 

Though the lesson here is obvious, it bears repeating: customer dissatisfaction can cost you significantly more than it does to fix defects before they make it to release.   Though, in retrospect, our client might have benefited from our Vendor Quality Management services when they hired their developer, they made a relatively uncommon decision in choosing to miss their deadline—a decision that delayed their $150m payday. but didn’t squander it.  We’ve seen more than our fair share of companies do the opposite—throwing quality to the wayside and letting the deadline drive the release—and pay for it dearly in lost future revenue.

Fortunately, our client knew better. And you can too—let us show you exactly how we can help you make a similarly-informed decision by calculating exactly how much a little investment in quality can be worth. In our client’s case, it meant that their customers got the Christmas gift they’d long been hoping for—even if it arrived a year late!

I don’t know about you, but I’d take that over a buggy, glitch-laden release any day of the week.  Wouldn’t you?


Mike Hodge
Lighthouse Technologies, Inc
Software Testing | Quality Assurance Consulting | Oracle EBS Consulting

{ 0 comments… add one }

Leave a Comment

PMIASQIEEESoftware Engineering InstituteInternational Software Testing Qualifications Board