Best Practices for Healthy Test Automation

It’s a new year and today health and wellness have become the new luxury and people all over the world are moving towards being healthier and fitter. That’s really awesome! and it triggered me to write an article about keeping the Test Automation healthy and fit. The first thing that comes to mind when we think about delivering quality software at a faster rate is Test Automation.

Test Automation is a software testing technique that uses special software to execute repeated tests against the target software in a safe environment, automatically running through the features of an application to ensure everything works as expected. It might appear to be a magic silver bullet to build a suite of Automated Tests, but in the long run, one of the biggest challenges is the Maintenance of your tests to respond to changes in the software. Jim Hazen puts it as “It’s Automation, not Automagic”. Yes, that’s true. Test Automation is not automagic, it is a software that tests another software and it needs to be maintained regularly to get more value from it.



Here in this article, I have shared some tips and tricks to reduce the maintenance burden and focus more on implementing new tests by having healthy test automation. These tips are based on our team's learning and my own learning. Some tips here might be well-known but there is a high chance that you will discover something new too.

Before diving deep let me summarize the tips and tricks here:

1. Stick to a simple and disciplined design pattern

2. Find the right areas of the application to be automated

3. Aim for keeping it all green

4. Make the tests as independent as possible

5. Be mindful while changing the state of the system

6. Choose the most robust and resilient locator for web-based automation

7. Avoid copied and pasted code instead focus on making simple reusable code

8. Be very cautious while writing code to delete test data

1. Stick to a simple and disciplined design pattern

Being in an agile environment there will be frequent software changes as new features are delivered constantly, hence it is very important to have a simple and disciplined design pattern that makes the automation framework more readable, easier to maintain and update whenever there is a new change. Here at Eagle Eye, our automation team uses one of the most popular design patterns Page Object Model (POM) for web-based automation. This pattern suggests, for each web page in the application, there should be a corresponding page class that contains the properties that represent the elements of the UI page and methods that interact with these elements.

Let me explain this with an example. Schemes is one of the modules in our AIR dashboard application, which contains two different pages Create New Scheme page and Search Scheme Page. In our automation framework, we have SchemesCreateNew and SchemesSearch classes which contain methods that perform all interactions with the corresponding pages. we also have SchemesHelper which is the base class for SchemesCreateNew and SchemeSearch class and it contains all the methods common for both the classes. On top of that, there is a trait called SchemesObjects which contains all the locators to the elements on the actual pages. Locators are the way to uniquely identify the web elements (such as input fields, text box, button, etc.) on a web page. Keeping all the locators in a single place eliminates the duplication of locators as well as it is easier to update in a single location whenever the locator is changed.

That's the simple design pattern we have for our web-based automation. Similarly, for our API framework, we have a simple pattern where each API has a corresponding class with all the endpoints related to that API and we have request and response classes to hold the request and response structures. Also, there is a separate base class specifically for hitting the APIs.

The trick here is to “Think more and code less”. More code means more maintenance, hence before writing any automation code it is a good practice to spend some time providing the right name to the class so that anyone can easily locate the class and figure out the right place to keep the code by sticking to the design pattern. This avoids code duplication and makes the framework more code friendly which in turn saves lots of time on maintenance work.

2. Find the right areas of the application to be automated

“It is Quality rather than Quantity that matters”, and this applies to test automation too. It’s often very easy to write thousands of tests in a short span of time and boast about it. However, if those tests are not well implemented and if it does not add clear value to the business then in the future it will increase the maintenance burden.

For example, it’s a bad idea to have thousands of tests for a feature that is only used internally by a small group of people. This will increase the maintenance work instead of adding much value to automation. In such scenarios, it is always good to have automation for the happy path scenarios which are more often used by the internal users.

Hence, it is very important to familiarize oneself with the application and understand who the end-user will be and how will they use the feature as well as find the business-critical scenarios before writing any new tests. Refinement sessions and review sessions give more insights on the new feature and so it’s important for automation engineers to be available during those sessions and gather those critical scenarios to be automated.

3. Aim for keeping it all green

One of the keys to healthy test automation is keeping it all green. Keeping it all green doesn't mean removing all assertions and never allowing the test to fail. It’s all about making every effort to resolve the test failures as quickly as possible when the application changes or when the automation code requires some improvements. So, if some tests fail, it means there is a bug in the application.

Sometimes the situation might be trickier, where some known bugs that will not be fixed in the near future might be failing the tests. To tackle such a scenario we came up with a process of creating backlog tickets for the failed scenarios and linking it with the original bug ticket. In the automation framework, we comment out the steps which are failing because of the known bug with the ticket details. This way, automation packs can be kept green and the commented out steps will never be missed. This also helps us to focus on the new failures instead of going through the known failures during regression.

4. Make the tests as independent as possible

Another key thing in healthy test automation is making the tests as independent as possible. Having dependencies in the tests makes it more complicated to read and it also increases the maintenance work. For example, let’s consider that Test Scenario 1, Test Scenario 2, and Test Scenario 3 are dependent on each other so if Test Scenario 3 alone fails we have to rerun all the tests which increases the execution time as well as making it very difficult to debug and fix the tests. If the tests are independent, it will be easy to re-run only the failed test and fix the test or raise an issue immediately without spending much time understanding the dependencies in the test.

Here you might wonder if making tests independent contradicts the statement of making tests simple without any duplication. Let me illustrate this with an example of promotional campaigns. Eagle Eye customers use our services to create promotional campaigns so let us consider there are two scenarios, the first one is ‘user should be able to update the newly created campaign’ and the second one is ‘user should be able to clone the campaign’. In this case, if you try to keep the scenarios simple without any duplication, you will create a campaign first, keeping the newly created campaign identifier in a static variable which will be used for the other two scenarios, inadvertently creating dependencies in the test. On the other hand, if you try to create a new campaign for every scenario in an effort to keep the tests independent you might end up with lots of duplication and execution time will be increased. To achieve both simple tests without duplication and independent tests, keep the campaign creation as a pre-requisite method and create the campaign with a unique name and try to fetch the campaign Id using the unique names and utilize them in the tests. This way the tests will remain independent of each other without duplication and the execution time will be reduced.

5. Be mindful while changing the state of the system

Sometimes we might want to change the state of the system to execute some tests. In such scenarios, we must ensure that the system is put back to the original state after the test execution. Be conscious while putting the system back to its original state and never keep those actions inside the test especially after the hard assertion statement. In the case of test failure, the system will never come back to the original state and it will affect all the other tests.

For example, one day we noticed that there were huge test failures, and upon root cause analysis, we understood that one of the tests had changed the timezone of the retail outlet and it never put it back to its original state due to some failure inside the test. This had caused all the other subsequent tests to fail. One of the lessons learned on that day is to never keep the actions that bring the system back to its original state inside the test. Instead, it is a good practice to always keep such actions at the end, independent of the tests.

6. Choose the most robust and resilient locator for web-based automation

The next big trick to keep the test automation healthy especially for web-based automation is focusing on the most robust and resilient locators like unique HTML element Ids. If that option is unavailable then look for a unique HTML name or class or links or text or partial link text. If none of the above options is unique then look for ancestors or descendants with unique attributes and traverse the Document Object Model (DOM) using a parent-child or sibling structure. In the case of dynamic locators just try to grab the stable part of the dynamic locator or figure out the nearest element with a unique attribute and traverse the DOM.

Sometimes the situation might be trickier where some parts of the element changes based on the environment and the stable part of the element may not be unique.

For example, in one environment if the web element is:

<div id="stablePartOne-987654-stablePartTwo"> some text </div>

and in the other environment it is:

<div id="stablePartOne-123456-stablePartTwo"> some text </div>

In this scenario, though the element is unique on the web page, it changes based on the environment. So creating a separate hard coded locator for each of the environments will increase the maintenance work in the future. The best way to tackle this is to divide and conquer the element. Divide the element Id into the stable part and the dynamic part and get the dynamic part from the database where all the test data is present. The simple locator will be:

locator = "'#stablePartOne-'.getUniqueIdentifier().'-stablePartTwo'" 

where the getUniqueIdentifier() method gets the dynamic part of the element from the test data.

7. Be very cautious while writing code to delete test data

One fine day, our team noticed that some test data being used for API automation was deleted, causing the tests to fail. On diving deep, we understood that some deletion code in the web automation framework deleted random test data instead of the intended data and caused tests to fail. The root cause was the deletion code didn’t perform null validation and entered a null value in the search field and deleted the first search result. Yes, that was horrible, but we fixed it immediately by adding null validation and checking whether the intended data is getting deleted. It’s a very good lesson to learn to be very cautious while writing code that deletes something from the system. Another tip here is to keep all the deletion code in a single helper class so that all the deletion codes can be accessed from one place.

8. Avoid copied and pasted code instead focus on making simple reusable code

It’s often tempting to quickly copy and paste test code with some small tweaks to quickly create a new test scenario. This might be a temporary win but over a period of time we will end up with lots of copied and pasted code which creates a lot of maintenance work. If there is a small change in the copied and pasted code then we have to spend more time searching and replacing all copied and pasted code.

To avoid this, start creating reusable methods instead of the copy and paste technique. Have all the common logic inside a single function and use that function wherever required. Also, if the logic is too huge, break them into multiple simple reusable methods and group those methods to your convenience. This will save lots of time on maintenance work and it leads to healthy test automation.

Let me take a simple example, if we want to make a convenient method to click search after entering from date and to date, we might be tempted to write a single function with all the logic inside it and make the code work:

<?php

function enterSearchAfterGivingStartDateAndEndDate($startDate, $endDate) {
    $startDate = date('Y-m-d H:i',strtotime($startDate));
    $endDate = date('Y-m-d H:i',strtotime($endDate));
    $this->type('#startDateElement', $startDate);
    $this->type('#endDateElement', $endDate);
    $this->click('#searchElement');
}

?>

Though it works fine, if there is a future change in the above lines of code or if there is a need for another convenient method like entering only the end date and clicking search then we will be again copy-pasting the above set of code with small changes. The better solution would be to create simple individual reusable functions and group those methods together.

<?php

function getDate($startDate) {
    return date('Y-m-d H:i', strtotime($startDate));
}

function enterDate($element, $date) {
    $this->type($element, $this->getDate($date));
}

function enterSearchAfterGivingStartDateAndEndDate($startDate, $endDate) {
    $this->enterDate('#startDateElement', $startDate);
    $this->enterDate('#endDateElement', $endDate);
    $this->click('#searchElement');
}

?>

One more bonus tip under this topic is always to keep assertions separate from logical methods. So that the logical methods can be reused for both positive and negative scenarios.

Wrapping up!

Let me wrap this up with a funny quote:

“Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.” - Martin Golding.

I hope these tips and tricks help in having a healthy and maintainable Test Automation. Thank you for reading.

Have a Happy and Healthy Test Automation!