As we know, DevOps is a very popular methodology in software development. It’s a set of practices and tools that support us to shorten the development lifecycle. While DevOps focuses on the integration of development and operation, TestOps specially focuses on the testing activities. In this article, let’s see what the TestOps components are and the key aspects of TestOps may include.
1.What is TestOps?
As we know, product quality is very important, and everyone is responsible for it. Therefore, not only the tester, but also the developer, project manager or stakeholder get involved in testing activities. It may lead to the complexity increasement. We also need to continue improving the speed of testing activities to deliver the product as soon as possible. Especially, if we have a huge system, our testing activities should be scalable. We need a set of practices and tools which help us maximize the effectiveness of testing activities. That’s called Test Ops.
2.TestOps components
TestOps can take part in important phases of testing activities, and they can be divided into these components.
2.1 Planning
Based on our testing experience, planning is an important activity. The output of this activity may be the testing type, testing level, how the application will be tested, the priority of the test case or the owner of each activity. In this phase, all the team including developers, business owner and tester work together the make decision. After that, everything will be shown on the tools to make it transparent. Besides, some AI tools also support us to propose the priority of test case or the test suite for testing.
2.2 Management
Test management will include the activities for managing resources, test environment, tools, process, and test plan. TestOps can help to make sure we can manage them efficiently, and the testing activities are scalable.
2.3 Control
This makes sure that we can retain the quality of testing based on the activities like creating policy for pull request and merging code, coding standards, code review or release process.
2.4 Insights
During the development lifecycle, we need a dashboard to show runtime status of the testing activities. It helps us to have an overview of the testing activities and the status of the product quality. The information shown maybe includes team and individual performance data, number of passed/failed test cases, the automation coverage, the test case coverage, or failure trends.
3.Test Ops key aspect
Based on the above components, we can define the key aspect of Test Ops
3.1 Automation testing
Certainly, automation testing is a must-have item in Test Ops. It can cover various kinds of testing levels like unit test, API test or UI test and reduce the human effort for test execution. It’s very easy to integrate the automation testing frameworks with the DevOps pipeline. With automation testing, we can run tests continuously so that we can have the status of the product early.
3.2 Continuous testing
In traditional development modal, testing is a separate phase after developing software. For now, testing involves many phases in the development lifecycle. Beside the main testing phase, we can see that we have shift left and shift right testing. Leveraging the CI/CD tool and automation testing, we can run the unit test or smoke test in the DevOps pipeline to make sure the quality of the code changes. After the deployment, we keep testing on the production environment to monitor the status of our product.
3.3 Code merging
The quality of test script is also very important because it’s one of the key points which decide the quality of testing activities. Without good quality, the test script can cause flaky test, wrong failure or missing bug. For improve script quality, we can define code review checklist or apply the code convention tools for scanning our script. We can define the policy for pull request and code merging like number of approvers, passing the smoke test, linking work item to the pull request. Until now, many tools support us to do that.
3.4 Test Result Analysis
After the test execution, analyzing the test result is very important and takes a lot of effort if we have a big project. We need to have an overview related to the product quality. It’s great if we have a dashboard to show the number of passed/failed test cases, the test coverage for each component or the root cause statistics of the failed test cases. Besides, leveraging AI for test results analysis can help us reduce the effort for this activity.
3.5 Test planning and management
The TestOps tools can support us to design the test cases and store information related to test case like priority, steps, or related features/bug. After that, we can easily create test suites/test plan on the UI. Not only the technical guys but also the non-technical guys can easily understand the steps of test case or purpose of the test suite or test plan so that they can review and give feedback on our test plan or test result.
3.6 Bug tracking
We also need to care about bug tracking activities. TestOps tools can support logging bugs quickly into the bug tracking system. Beside that, the bug statistics are important for us to have the overview related to product quality and make decisions for release. The bug statistics can include information like bug severity, bug priorities, number of re-open bugs or side effect bugs. It’s also a fact to measure the quality of the development team.
3.7 Collaboration
Like DevOps, TestOps will increase the collaboration of the development team and stakeholders. The TestOps tools can show us the owner of the test cases or testing activities. They also show us the status of the testing activities so that we can easily manage progress and support each other.
3.8 Monitoring and observation
Building a monitoring and observation system to track the performance of application in the real time will help the development get the feedback early. We need to monitor the CPU, memory, storage of the server continuously to make sure that our services are highly available. The tools also support us to run simple workflows periodically to make sure that our product works well.
3.9 Environment management
To make sure that our test results are correct and consistent, testing environment is very important. We need to make sure that our test environment is the same with the defined test strategy. With the big system, it requires our test environment to be scalable, especially for performance testing or crossed browser testing. By leveraging the TestOps tools, we can create isolated test environments, initialize them or create more testing instances quickly.
Conclusion
Adopting TestOps will help us reduce the time it takes to release our product to market and save our costs for software development. However, we need to care about the team culture, the project budget and suitable tools for applying TestOps in each phase of our development lifecycle.
Reference:
- https://en.wikipedia.org/wiki/TestOps
- https://insights.daffodilsw.com/blog/what-is-testops
- https://www.accelq.com/blog/what-is-testops/