top of page

Advice #2: Make Your Tests Independent Even If They Fail

Updated: Apr 3

Your test cases should be independent. However, instead of setting postconditions that will

not be executed when a test fails, add an appropriate precondition that should be set at the beginning of the tests. In this way, your test cases will be fast and independent even if

some of them fail.

An important requirement in test automation is that the test cases should be independent. If so, you can execute them in parallel or in any order. If not, then some test cases may fail because they are faulty not because the system under test is wrong.

To make test cases independent, one solution is to set an initial environment for each test case and start the execution from here. Unfortunately, this solution can be very slow. Setting the initial environment usually requires a file copy, then there may be several steps to be executed to set the required state from which the feature can be tested.

To avoid repeating the same steps many times, when a test case has been executed a postcondition can added, to set the necessary state for the subsequent test case. The problem with this solution is that if a test case fails, then the required state will not be set and subsequent tests will also fail as the initial state for the subsequent test is faulty.

The solution is that instead of setting the initial state by applying a postcondition, let’s set the initial state for the requirement for each test case as a precondition. For example, instead of starting the application and logging in for each test case, a better solution is to check whether the application is logged in at the beginning of each test. This is a simple validation and if so, the execution starts. On the other hand, the test starts with the login process.

To do this a condition should be added at the beginning of each test. Be very cautious as conditions in the test cases do not always work. Before the condition is executed you should validate something. For example, you can check that you are logged in, i.e., your name is visible.

Harmony has an excellent solution for setting the necessary states for a given feature. You can set a state at any point in your model. This state can be a start state for other test cases. Here is an example:

In this figure the initial state is to log in a user, then an action happens where we go to the Parameters window. The next action includes the condition, but before this, we should check something assuring the necessary waiting time. The conditional action is to delete a ’prod’ environment when it exists. Finally, we add a state where there is a single environment exists.

The „delete ’prod’ when exists” action consists of the following executable steps:

The first step is the condition, checking that the environment ‘prod’ is visible. If not, then nothing happens. If so, then we click on the ‘prod’ environment, and then the three dots next to it. Now we can select deletion and by clicking ‘OK’ it’s deleted. The selector name for OK is OK 3 as there are more OK buttons in this feature.

Now this state ‘SINGLE ENVIRONMENT’ can be used for test cases to test the feature Parameter handling. Independently if the environment ‘prod’ exists or doesn’t exist the initial state will be correct even if any test fails.

Here is a model that consists of two test cases. Both start with the initial state ‘SINGLE ENVIRONMENT’. In the second test, we add environment ‘prod’, in the first one we don’t. We should remove ‘prod’ otherwise both test cases would fail for the subsequent executions. Fortunately, the initial state assures this, independently of the environment ‘prod’ exists or doesn’t exist:

The consequence is that by applying states and conditions, you can make your test cases independent and fast, even if any test case fails.

44 views0 comments


bottom of page