Modern test automation is about streamlining the process of verifying that web applications work correctly. In this article, I explain what are the latest trends to watch for.

Introduction

Test automation is vital when you are building any web-based application. It allows you to test the core functionality quickly, efficiently, and automatically. However, certain types of content make test automation challenging. For instance, GIS information, such as maps, weather data, or live tracking data, are nearly impossible to test using classic test automation. Modern test automation seeks to address these challenges using AI. Here, we explore the current state of the art for machine learning test automation tools and look at some future trends. 

Why is GIS data hard to test?

Test automation is usually pretty dumb. You connect your website or application to a special environment called a Web Driver. This allows you to control what happens on screen via a script. This script tells the Web Driver which elements to select, how to interact with them, and what the expected outcome is. For instance, it might tell the Web Driver to find the login form, enter an email and password, and then click “Sign in”. 

There are a couple of real problems here. Firstly, the Web Driver is operating in what is called the main DOM (standing for domain object model). Any embedded content is effectively invisible. For instance, if you have an embedded Google Maps object, or a live weather radar image. At best, you can tell the test script to check that something is there, but not that the right thing is there. Secondly, even small changes to the application or site code can cause the script to select the wrong object. This causes numerous false positives, where a test seems to fail but there is no actual bug present.

What is the state of the art?

For ages, test automation stagnated, just seeing incremental improvements to the existing approach. However, in recent years we have seen a trend towards AI-powered test automation. So, how do ML-powered test automation tools work?

Intelligent test recorders. These create tests by navigating through your site like a normal user. Under the hood, the system is learning what the test actually does and builds a complete model of your site. The advantage here is the system gets to see your entire site, including embedded content. That makes testing embedded GIS data much easier.

Self-healing tests. Here, tests use machine learning to adapt as your UI evolves. Elements are identified intelligently in the same way a human user would. As a human, you know that “Purchase”, “Add to Cart”, and “Buy Now” mean the same. A machine learning test automation tool knows this too thanks to the power of natural language processing (NLP). This slashes the number of false positives you will get when your site is updated.

Visual testing. One weakness of test scripts is they only test what they’re told to. That means lots of the UI is left untested. Quite apart from the embedded content they can’t even see. Nowadays, computer vision allows for much more intelligent testing. You can tell the system how much of the site is allowed to change visually. For instance, you know that an embedded map will always have certain geographic features visible. If it misloads, these won’t be  there and the system can warn you something is wrong.

Trends to look out for

AI is already transforming test automation, thanks to a combination of machine learning, computer vision and NLP. But what about future trends?

Customer-journey driven testing. How do you know which tests to perform on your application? Usually, you speak to the product manager and design team to understand how they think the app will be used. But users seldom behave exactly as expected. That’s when you need to use real world data to understand how users actually interact with your application. Then you can compare this with the tests you run to identify any gaps.

In-production testing. Most applications embed content or functionality from other providers. This is particularly true for GIS data. The problem is, how can you tell if the integration is working smoothly in production? The answer is, you need to continuously run your (intelligent) automated tests against your production system. These tests are able to spot when something isn’t right and can alert your DevOps team.

Autonomous testing. The ultimate dream for ML test automation tools is autonomous testing. Here, no human input is needed at all. Your test system evaluates the application, decides what needs to be tested, and creates the required tests. It then automatically runs these tests any time they are needed and gives your CI/CD tool a release/no-release decision.

Conclusions

Test automation has rapidly become the most intelligent part of the software development process. This is thanks to test automation platforms that leverage machine learning and other forms of artificial intelligence. These platforms overcome the limitations of traditional test automation. They can cope with the most complex of embedded content. And the tests are robust against false positives triggered by site updates. Before long, the platforms will be so advanced, they will be able to create and run tests autonomously.