Artificial Intelligence is changing every aspect of the world and so does software testing. The ability to learn from mass of historical data to recommendation making testing faster and highly automated. Currently, with MCP, there’s an open windows for more advanced workflows like:
- Automatically generate test cases in test management system such as Azure DevOps.
- Automatically run automation test script via prompt for example using Github Copilot and Playwright MCP.
- Automatically detect UI changes and suggest updates
- …
So in this blog post, we will have a deeper look at how we can leverage AI to refine software testing to make software deliverable faster, efficient and reliable.
THE NEED OF INTRODUCTION AI INTO SOFTWARE TESTING
The raising of deliver software flawlessly and faster is higher than ever. So, with traditional testing, where we mainly do manual testing and automation test for regression will sooner get trouble with complex and often changed system in current Agile world. There are a lot of test cases, test scripts to create, to update in a short time with limited resource. However, with the help of AI, this can be much easier and faster. AI can also give support in all stages of testing, from test approach preparation, test case, automation test script generation to reporting and analyzation.
Test approach generation
Creation testing documents is easier in AI’s age. Just with the template, standard samples if any, and all of the project’s requirement, AI can help to create test approach in minutes following company’s template.
Test case / test script generation
Traditional, QC reads the requirement and prepares test case, then creates the automation test script. As coding, QC has to prepare the framework and needs QC to have coding skills.
However, with the help of AI, with just some natural language prompt, test case and test script can be created faster. The created test case then can be automatically imported into test case management system. For example in these below screenshots, Playwright MCP and Github Copilot help to create automation test script in just minutes.
Test data generation
In software testing, we need a lot of test data that simulated the real situation. However, to protect sensitive data, we can’t use real client’s data and those data might be not enough for testing, too. Besides, sometimes, it might be difficult to create test data such as for autonomous car, to test the obstacle recognition, such as a building, you have to test a building in day light, night life, at noon, rain, color, with other objects such as human, tree, cars, …. To take photo of those scenarios is not realistic, and creation synthetics data is a good approach and AI can help.
In the screenshot are test data for realistic address in UK created by ChatGPT, for example.
Issue identification and fixing suggestion
Not only helping in creation test case, automation test script, test data, AI can investigate and suggest fixing when there are error in the script, … such as in below screenshot.
Continuous Testing in DevOps and CI/CD
For big system with historical data, such as bugs, test report, quality KPIs, …, we can use AI to analyze and recommend focus testing on high risk areas, reduce waste-time on low-risks areas. Then it is integrated into DevOps and CI/CD to automatically run test whenever there is code change to know immediately feedback. This makes the deliver cycle faster and more accurate.
Test maintenance
When there is a change in the requirements or in the code itself, QC have to update the test scripts and this costs a lot of time as in today’s business, frequent changes and updates is a must.
However, with some AI testing tools, there is self-healing function so that it can automatically update the script to adapt with changes. This will
For example, instead of using traditional locator like XPath, id, name, … like below
Using AI testing tools, you can write “enter “glass” into search box” then when the UI changes or code changes, AI tool will look for search box and automatically update the script.
CHALLENGES OF USING AI IN SOFTWARE TESTING
Although AI can do such kinds of tasks in software testing, however, normally they are used in key activities such as test case, automation test script, test data generation and we still can’t dependent 100% on AI tool. AI’s testcase, test script, … still need human’s on verification and update. It is also challenging for AI when working with complex requirements and code base.
SO, SHOULD WE USE AI IN TESTING?
AI is revolutionizing testing by improving efficiency, accuracy. While it may not replace manual testing entirely, it becomes an essential tool for testing and its role is growing. So to stay ahead of the competition, we have to evolve AI in our testing process.