top of page
Search

Advice #7 (last advice): Use MBT to Make Test Automation Efficient

Updated: Apr 17

Test automation transcends mere execution; it’s a fusion of test design automation and test execution automation. The key to unlocking its full potential lies in Model-Based Testing (MBT). Traditional MBT, while effective, isn’t without its limitations. To overcome these, we’ve pioneered an innovative approach: Two-Phase MBT coupled with Action-State Testing.



For lots of companies, test automation is restricted to test execution automation. This means that the test cases are coded one by one without the necessary test design. The result is the inefficient test cases and the poor code quality. Another possibility is to make the test design separate from the test automation. In this case, test design and automation should be maintained in parallel. However, after a while, the two will be separated forever.


The obvious solution is to unite test design and test implementation into a single process and tool. According to Bloor research: ‘Test design automation is the ability to automate the entirety of the testing process starting at an extremely high level’. The best solution for test design automation is model-based testing (MBT). Here we start from the requirement specification or some similar available description of the planned system and make a model.


The model can be graphical or textual. Most models are graphical and even textual models can be converted to graphs. Based on a test selection criterion such as the 0-switch criterion the (semi-)abstract test cases are generated. From these test cases, the executable test code can be generated.


The model is more compact than the code, thus it is faster to create. While the test cases may contain the same test step more times, a good model consists of a step only once. When a requirement changes, so is the model, and the test code is generated again. Therefore, no code refactoring is necessary. Instead of obsolete tests, obsolete model elements remain that are easier to fix.


If there are so many advantages of using MBT while most companies remained in the past using traditional tools, frameworks, and libraries? The answer is that traditional MBT has some disadvantages as well.  One problem is that the model should be readable for the application to parse and process the model to create executable test code. The result is a model that consists of too many elements making it less understandable. The other consequence is that the model can be created when the application under test is ready, otherwise, the model and the application will be different.


Another problem is that you cannot avoid coding. When the model graph is traversed some invalid test cases may be generated. To avoid this problem, the so guard conditions are inserted such ‘numOfTickets <= 4’.   However, coding may involve bugs.


The third problem is handling the outputs. Different outputs can be present along different paths, therefore, traversing a path should consider the related output, i.e. simply listing the test steps is not enough. There are different solutions but none of them is perfect.


Finally, the models are either stateful or stateless. Unfortunately, the requirements determine which one to choose. However, the current methods and tools are either stateful or stateless, and if you make a stateless model for a system that requires states, then you will not detect several potential bugs. In the reverse case, you should insert unnecessary states making the modeling more difficult.      


Fortunately, a better modeling technique, action-state testing addresses some of these problems. In this model, a model step comprises a user action (input), none, one, or multiple system responses (outputs), and an optional state. To eliminate the need for guard conditions, the model is composed of scenarios, which can fork and join. These scenarios may also include unique outputs. The modeling can be done by using a text editor, yet the model is a graph annotated with the test steps. Here is an example:



Another technique is two-phase MBT. First, a high-level model is created that is implementation independent, thus can be done before the application has been implemented. This model is understandable for humans only. For example, in the model above the action in the third line ‘add pizzas to reach EUR 30’ is understandable for a tester and she can execute it in several ways. When the application is ready, the tester executes the steps described in the model one by one. During the manual execution, the low-level model and the automated test step are generated. Any failed step can be deleted and do it again. A key advantage is that no test debugging is necessary. To learn more, you can read our book or some of our former blog posts.   


Modern MBT doesn’t not only make the test automation time significantly shorter and detects more bugs, but makes maintenance more efficient and comfortable. Companies using better techniques will win the others will lose. In Harmony, we applied all the pieces of advice. Here is a summary of the functionalities our tool has while others haven’t.  


  • Flexible Modeling Options: Offers both stateful and stateless modeling to cater to different system requirements.

  • One-Click Negative Test Case Inclusion: Streamlines the addition of negative scenarios.

  • Unique ID and Special Attribute Resolution: Harmony uniquely addresses the common issue of non-unique IDs and special attributes, allowing testing to continue uninterrupted.

  • Partial Execution of Failed Test Cases: Let’s mark the last test step passed that ensures all tests can pass.

  • Automated Flaky Test Recovery: Our innovative method resolves flaky tests automatically, saving time and resources.

  • Effortless Selector Modification: With a single click, Harmony adapts to changes in selectors, greatly simplifying maintenance.

  • Obsolete Test Case Remediation: Our requirement traceability feature accelerates the updating process, making it faster and more efficient.

  • Custom commands: Users may need specific actions or assertions. These require coding. However, with our prompts, any necessary code can be generated by Microsoft Copilot in Bing.


Summarizing, we suggest using model-based testing. You will see that instead of boring test maintenance, you can do interesting test design and modeling. A final piece of advice is if you haven’t read our previous pieces of advice, let’s do it. In case of any questions or comments don’t hesitate to contact us. 



15 views0 comments

Recent Posts

See All

Comments


bottom of page