Artificial Intelligence: evolving software testing

The term Artificial Intelligence (AI) brings up all kinds of fantastic images to mind, thanks to Hollywood. We have robots of both evil and kind persuasions and of course, they have human forms, because that’s what sells. All of that being said, what about real life?

The robots in the movies go beyond what they are programmed to do and have the ability to reason and to learn. In reality, we have robots programmed to do simple building tasks in industry and continues to expand into other areas of usefulness. One great example we have of artificial intelligence objects in our everyday lives are tools like Alexa and Siri. They are programmed to learn our preferences, or at least keep in memory what we like. The question and answer exchanges make them “feel” like they are learning our entire lives, but what they really are doing is responding to a series of programmed responses to anticipated triggers.

Now let’s turn AI to software development testing. Is it really a thing? The answer is a solid YES, and just like AI itself, the proficiency and capabilities of AI continue to grow each year. The growth of AI goes with a familiar flow of software development.


The spectrum for testing software has never been so vast. Applications today interact with other applications through APIs. They leverage legacy systems and continue to grow in complexity from one day to the next. Now, the next question is…what does that mean for testers?

The 2016-17 World Quality Report suggests that AI will help. “We believe that the most important solution to overcome increasing QA and testing challenges will be the emerging introduction of machine-based intelligence,” according to the report.

Change is here

Things you should expect to see changes in software testing as a result of AI are:

  1. Accelerated Process
    Manual functional testing is expensive, in time and funding. The test scripts are written line-by-line, as simple and redundantly as possible, requiring explicit attention to detail. This may take anywhere from several days to several weeks depending on the complexity of the program. AI can sort through log files, saving time and enhancing accuracy. What AI finds gives QA Engineers a complete view of what changes they must complete.

  2. Increased test coverage
    The overall depth and scope of tests resulting in an overall improvement of software quality increase with AI. Automated software testing can look into memory and file contents, internal program states and data tables and determine if the software is behaving as expected. In fact, test automation can execute 1000+ different test cases in every test run providing coverage that is not possible with manual tests, without the requirement of an army of testers.

  3. Process Automation
    Hardcoded test scripts are fragile in a sense. They work as they have been originally designed, however, the slightest change requires maintenance. As a result, a QA Engineer needs to review and update the test code to account for every change. This creates an additional need for resources that lead to additional expenses for testing.

    AI bots recognize changes in code. They are able to adapt and learn new application functions. Once modifications are identified, AI decides if it is new functionality or defects resulting from a new release. This greatly reduces the need for QA Engineers to modify the test scripts for every minor change in code.

  4. Defect Reduction
    Even when the QA Engineer does everything correctly in writing the script, a bug can still slip through the testing event, unnoticed. This may be the result of an unknown variable, a missed part of the script, or human error. When this occurs, analysis (investigation) is necessary to determine how and why the defect was not detected. This adds significant time to the testing event and remains inevitable on even the strongest teams.

    AI eliminates the user-error scenarios on the testing end and is able to conduct ongoing analysis of errors in a fraction of the time it takes the QA Engineers to complete their analysis.

  5. Improved accuracy
    Even the most attentive tester experiences fatigue with monotonous manual testing. With this fatigue, errors are bound to happen because…. humans are imperfect. AI handles repetitive testing fast and flawlessly every time.

  6. Reduced need for live resources
    AI used with automated testing can simulate the volume of testing that once required hundreds and even thousands of live users to achieve and can interact with the network, software, or web-based applications.

  7. Developer assistance
    Tests can run automatically with every source code change and notify developers if they fail.


What about the humans?

\"\"While the current practices of Agile, Continuous Testing and DevOps are keeping the software development process at pace, unlocking the true potential can only happen by leveraging the power of Artificial Intelligence in software testing.

Since expectations from testers are changing, the very nature of the work that the testers are performing is undergoing changes as well. This leads to an important conclusion: While it’s unlikely the testers will be wiped out, machine learning and other branches of AI will significantly alter the way software testing is conducted.

The job of a software tester involves taking data-backed decisions to ensure the quality of software meets the set standards. Leveraging AI for the same task can result in software testing taking place at a much faster pace while managing a larger amount of data. Application of AI to testing enables the testers to go beyond the traditional mode of testing and adopt AI-enabled automation platforms instead. Exploratory testing is likely to remain a human testing task being that AI cannot mimic the creativity of this kind of activity.

Robots lack the layered reasoning it takes to explain why a certain format is preferable over another, why certain objects on the screen should be big, small or a particular shade of turquoise. This is where usability testing will continue to require major human inputs. Whether the navigational flow is intuitive or user-friendly remains dependent on intelligent (human) observations. Consider an error message that pops up on your screen: A robot can see if a message is grammatical, but not if it is appropriate. As user technology becomes more intricate, software testing will continue to require common sense and empathy, and that will remain a human quality.

Lighthouse Software Testing keeps a close watch on the latest cutting-edge improvements in software testing as a commitment to continuous improvement. We provide our clients with the capacity of large-scale testing with the attention and care of a tight, well-connected team. Contact us to learn how we can launch your testing processes forward to reduce costs, increase accuracy and maximize efficiency.

{ 0 comments… add one }

Leave a Comment

PMIASQIEEESoftware Engineering InstituteInternational Software Testing Qualifications Board